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ABSTRACT 


This  thesis  demonstrates  the  advantages  of  tailoring 
the  representation  of  machine  instructions  and  data  to  the 
distributions  of  usage  found  in  a  computing  environment. 
Variable-length  encodings  can  save  memory,  and  at  the  same 
time  eliminate  overflow  in  all  its  forms.  We  identify 
common  dependencies  and  contributors  to  redundancy  in 
machine-language,  and  develop  techniques  for  measuring  and 
removing  them.  Using  these  techniques  on  a  sample  of 
machine-language  compiled  from  a  high-level  source,  we  are 
able  to  trim  75%  from  the  space  taken  by  a  contemporary 
machine  representation,  and  60%  from  the  space  taken  by  a 
language-directed  machine  representation  that  did  not  employ 
our  techniques.  The  gain  in  space  will  be  accompanied  by  a 
reduction  in  execution  time  due  to  more  efficient  use  of 
available  bandwidth.  The  cost  is  an  increase  in  hardware 
complexity.  We  present  a  range  of  data  organization  models 
that  save  space  and  greatly  reduce  the  probability  of 
overflow,  and  give  example  evaluations  of  two  models.  We 
evaluate  one  model  with  actual  data  to  find  the  nest 
representation  of  variables  within  the  model. 
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1  INTRODUCTION 

The  digital  computer  designer’s  world  is  a  discrete 
world  --  discrete  operations  on  discrete  information  units 
occurring  at  discrete  ticks  of  the  clock.  As  a  design 
proceeds,  the  designer  must  decide  what  quanta  of  space  and 
time  to  allot  each  of  the  functions  of  the  machine.  A 
common  design  decision  is  to  represent  a  class  of  values  by 
a  fixed  number  of  bits,  say  b.  This  decision  gives 
immediate  rise  to  two  problems:  some  users  are  bound  to 
want  values  that  are  not  in  the  finite  set  (of  at  most  2**b) 
provided;  and  the  values  that  are  provided  will  not  be  used 
equally  often,  so  the  fixed-length  representation  will  waste 
space  (Shannon  49) .  These  two  problems  are  complementary  in 
the  sense  that  changing  b  to  ease  one  of  them  will  aggravate 
the  other.  Since  it  is  a  greater  inadequacy  to  have  no 
representation  at  all  than  to  have  a  wasteful 
representation,  b  is  chosen  large  enough  to  represent 
'•almost  all”  of  the  values  that  might  arise  during  the 
computations  of  the  intended  class  of  users,  and  the 
efficient  use  of  space  suffers  accordingly. 

All  the  graphic  representations  of  numbers  that  people 
have  devised  have  been  variable-length,  as  indeed  any 
representation  (using  a  finite  character  set)  of  the  members 
of  any  infinite  set  must  be.  Why  then  do  computer  designers 
so  often  choose  a  fixed-length  representation?  First, 
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becaase  it  is  simple.  Conceptual  simplicity  makes  the 
design  easier.  Second,  because  the  hardware  for  a  simple 
design  may  be  less  expensive  to  manufacture.  These  oLear 
and  obvious  benefits,  along  with  the  assumed  benefit  of 
greater  speed  from  simpler  devices,  have  often  caused 
designers  to  decide  in  favour  of  a  fixed  number  size.  The 
benefits,  however,  when  properly  calculated,  are  only  aalf 
of  the  evaluation.  This  thesis  will  examine  the  other  half: 
the  cost. 

Figure  la  shows  the  frequency  distribution  of  some 
numbers  in  the  computing  world.  Figure  1b  shows  the 
distribution  for  which  a  fixed-length  representation  is 
suitable.  How  poorly  1b  approximates  la  is  apparent  at  a 
glance.  Recognizing  this,  probably  by  intuition  rather  than 
by  measurement,  some  designers  provide  two  or  three  number 
sizes.  As  figure  1c  shows,  this  goes  part  way  towards 
matching  the  natural  variability.  Unfortunately,  the 
programmer  or  compiler  is  required  to  decide,  for  each 
variable  and  constant  in  a  program,  which  of  the  three  fixed 
sizes  it  should  be,  and  hope  that  enough  space  has  neen 
taken,  and  not  too  much  wasted. 
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(a)  Distribution  of  Numeric  Constants  in  XPL  programs 

(Alexander  72  table  3.5.3) 
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Overflow  and  waste  are  certainly  not  restricted  to 
number  representation.  All  instructions  and  all  data  face 
this  problem  pair.  The  longer  items,  such  as  character 
strings,  arrays,  and  sequences  of  instructions,  iave 
received  more  attention  in  this  respect,  usually  from 
software  designers,  than  numbers,  operation  codes,  and 
addresses  have  from  hardware  designers.  PL/I,  for  example, 
allows  the  programmer  to  select  a  length  (up  to  some 
implementation-imposed  maximum)  for  each  character  string 
variable,  so  that  it  can  occupy  the  shortest  space  that  is 
long  enough  to  hold  any  string  value  it  might  take.  Dnce 
again,  as  with  numbers,  the  programmer  must  decide  what  this 
length  is.  The  overflow  problem  for  strings  is  more 
obviously  one  of  storage,  i.e.,  trying  to  stuff  a  long  item 
into  a  short  space,  than  is  the  problem  for  numbers.  There 
it  might  seem  to  be  a  problem  of  addition  or  multiplication, 
since  on  some  machines  an  interrupt  occurs  during  those 
operations  and  not  during  the  store  operation.  It  is 
really,  however,  a  storage  problem.  Even  floating-point 
underflow  is  really  a  lack  of  space  in  the  exponent  field. 

Some  systems,  especially  those  for  string  manipulation, 
such  as  SNOBOL,  consider  length  to  be  a  property  of  the 
value  of  a  string  variable,  rather  than  of  the  variaole. 
They  free  the  user  from  choosing  a  length,  solve  the 
overflow  problem,  and  they  don't  provide  more  space  foe  a 
string  than  it  actually  uses. 
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Machines  today  commonly  have  one  or  two  operation  code 
sizes,  and  one  or  two  address  sizes.  The  overflow  and  wist a 
problems  caused  by  fixing  these  basic  quantities  can  be  more 
serious  than  for  data.  Fixing  the  size  of  an  address  field, 
for  example,  limits  the  amount  of  addressable  memory  that 
can  be  placed  on  a  machine.  So  it  is  made  large  enaigh, 
hopefully,  to  satisfy  all  users  for  awhile,  but  no  larger:, 
or  too  much  space  would  be  wasted  in  every  address. 
Similarly,  fixing  a  field  that  addresses  input/output 
devices  limits  the  number  of  devices.  Quite  commonly  the 
demand  for  more  memory  and  devices  has  exceeded  the  design 
limitations,  and  the  whole  machine  had  to  be  redesigned  to 
add  some  memory  or  devices  (e.g..  Burroughs  B5500-B6 50 0- 
B6700  and  IBM  704-709,  7090-360).  Furthermore,  as  studies 
of  program  locality  have  shown  (Denning  70) ,  programs  do  not 
address  all  of  memory  evenly,  with  equal  probability  for 
each  address.  Fixed-length  representations  of  addresses 
take  no  advantage  of  this  fact,  although  it  represents  a 
potential  saving  in  space.  The  situation  is  similar  for 
operation  codes,  and  any  other  aggregate  of  bits. 

The  problems  of  overflow  and  waste  are  not  the  products 
of  a  particular  representation  of  information;  they  are  the 
result  of  quantization.  Synchronous  machines  quantize  time 
into  units  long  enough  to  do  basic  machine  operations;  e. g. , 
add,  divide,  access  memory.  In  the  simplest  version,  all 
operations  are  given  as  long  as  the  longest  operation. 
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Overflow  occurs  if  a  new  operation,  taking  longer  than  the 
basic  quantum  of  time,  is  added  to  the  machine.  This  may 
simply  require  lengthening  the  clock  cycle,  or  it  may  be 
almost  impossible  if  many  operations  depend  on  the  c^cle 
time.  But  the  real  cost  of  quantizing  time  is  the  wasted 
time  for  short  operations.  Sophisticated  synchroaous 
machines  use  a  quantum  of  time  that  is  long  enough  only  for 
the  fastest  operations,  and  allow  the  longer  operations, 
such  as  memory  access,  some  number  of  quanta.  While  this  is 
an  improvement  in  speed,  the  designer  is  burdened  with  the 
responsibility  of  giving  each  operation  enough  cycles,  and 
not  wasting  too  many  --  a  situation  similar  to  that  for 
numbers  and  PL/I  strings.  Asynchronous  machines,  suoa  as 
the  PDP-11,  accept  the  variability  --  each  operation  is 
given  just  the  time  it  requires. 

Now  that  we  have  identified  the  symptoms  of  fixing  *hat 
is  conceptually  variable,  we  must  discover  the  ills  of 
allowing  variables  too  much  freedom.  If  the  length  oE  an 
instruction,  address,  or  data  item  is  fixed  a  priori,  then 
no  bits  are  needed  to  describe  its  length.  If  the  time 
given  to  an  operation  is  predetermined,  then  no  signal  is 
needed  to  indicate  its  completion.  For  any  given  maoiine 
function,  a  variable-length  operation  time  is  worthwaile 
only  if  the  difference  between  the  maximum  operation  time 
and  the  average  operation  time  exceeds  the  time  required  to 


-7- 


detect  its  completion.  Similarly,  space  variability  is  not 
free,  and  may  not  be  worthwhile. 

In  this  thesis  we  shall  exclude  time  guantizations, 
considering  only  implications  of  space  guantizations.  For 
this  purpose  we  shall  divide  space  into  instructions,  which 
includes  all  that  remains  constant  during  program  execution, 
and  variable  data,  which  includes  all  that  is  subject  to 
change. 

Each  design  decision  is  affected  by  many  other  design 
decisions,  and  cannot  be  made  in  isolation;  each  new 
decision  causes  re-evaluation  of  old  ones.  This  thesis, 
however,  is  not  about  a  particular  design,  but  about  an 
aspect  of  design  procedure.  We  shall  consider  each 
guantization  separately,  and  indicate  the  related  decisions. 

This  thesis  demonstrates  the  advantages  of  tailoring 
the  representation  of  machine  instructions  and  data  to  the 
distributions  of  usage  found  in  a  computing  environment. 
The  contributions  of  this  thesis  are: 

1  the  identification  of  common  dependencies  and 

contributors  to  redundancy 

(a)  within  machine  instructions 

(b)  between  machine  instructions 

(c)  between  instructions  and  data 

(d)  within  data 

2  technigues  for  measuring  the  amount  of  redundaacy 
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3  techniques  for  removing  redundancy 

4  data  organizations  that  take  advantage  of 
dependencies  between  and  redundancy  in  the  vaLues 
of  variables. 

In  Section  2  we  present  the  background  for  the  thesis, 
including  the  history  of,  and  motivation  for,  variaole- 
length  encodings.  Section  3  gives  a  theoretical  foundation 
to  the  work,  including  mathematical  definitions  of  the 
terminology  of  the  thesis,  and  determination  of  a  Lower 
bound  on  space  requirements.  The  encoding  of  instructions 
is  the  subject  of  Section  4,  and  the  encoding  of  variable 
data  is  the  subject  of  Section  5.  In  Section  6  we  put  some 
of  the  ideas  of  the  preceding  sections  to  a  practical  test 
by  applying  them  to  an  actual  computing  environment.  We  are 
able  to  trim  75%  from  the  space  taken  by  a  contemporary 
machine  representation,  and  60%  from  the  space  taken  b/  a 
language-directed  machine  representation  that  does  not 
employ  our  techniques.  The  cost  of  a  data  organization  that 
greatly  reduces  the  probability  of  overflow  compares 
favourably  with  a  conventional  organization. 
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2  BACKGROUND 


Early  machine  designs  seem  to  show  more  concern  with 

possibility  and  reliability  than  with  efficiency. 

Simplicity  was  important,  and  that  meant  uniformity  of  data 

path  widths,  adder  width,  data  lengths  and  instruction 

length.  A  single  instruction  format  was  chosen,  with 

subjective  or  ad  hoc  justification. 

In  retrospect  one  wonders  whether,  in  each  choice, 
fitting  instruction  words  to  a  desired  data-word  length 
was  not  just  as  strong  a  factor  as  the  intrinsic  merit 
of  the  instruction  format  which  gave  rise  to  so  much 
discussion.  The  distinction  is  mainly  in  whether  one 
chooses  to  write  related  pieces  of  information 
vertically  on  a  sheet  of  paper  or  horizontally.  There 
was  remarkably  little  difference  among  most  of  the 
early  computers  with  respect  to  the  operations  that 
they  performed.  (Buchholz  62  p . 123) 


2.1  History  of  Variation 


When  the  IBM  704  was  redesigned  with  new  technology, 

more  addressing  space,  and  more  functions,  one  of  the 

resulting  machines,  7030,  had  two  instruction  lengths,  32 

and  64  bits,  compared  to  the  36  bit  instructions  of  the  734: 

It  is  obvious  from  information  theory  that  instructions 
of  varying  information  content  can  be  expressed  by  a 
varying  number  of  bits.  It  is  not  so  obvious  that  the 
saving  in  memory  space  for  programs,  which  results  from 
having  multiple  instruction  formats,  would  alone  pay 
for  the  additional  equipment  cost  of  decoding  these 
formats.  What  really  prompted  the  introduction  of 
multiple  instruction  formats  was  the  observation  that 
the  speed  of  the  7030  was  in  danger  of  becoming 
severely  limited  by  the  time  taken  to  fetch 
instructions  from  memory  during  the  execution  of  the 
all-important  inner  loops  of  arithmetical  programs.  At 
that  point  in  the  design,  it  was  found  that  almost  all 
the  instructions  usually  needed  in  the  inner  loops 
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(floating-point  arithmetic,  indexing,  and  branding) 
could  be  expressed  in  terms  of  32-bit  half  words  and 
that,  if  they  were  so  expressed,  the  number  of  accasses 
to  memory  for  instructions  could  be  cut  almost  in  half. 
(Buchholz  62  p.  128) 

Although  information  theory  is  invoked,  its  application  was 

superficial.  Buchholz  continues: 

An  analysis  of  the  operation  codes  shows  that  only  3.05 
bit  of  information  is  left  unused  in  the  32-bit 
formats.  The  0.05  bit  actually  represents,  at  the  time 
of  writing,  unallocated  space  in  the  formats  for  three 
more  floating-point  operations  with  their  modifiers  and 
nine  more  miscellaneous  operations,  each  with  a  fulL  li¬ 
bit  I  address,  which  is  not  a  trivial  amount  of  space. 
The  full-word  formats  are  not  so  closely  packed.  The 
64  bits  are  found  to  contain  almost  6  bits  of 
redundancy. 

with  the  footnote 

This  assumes  that  all  defined  combinations  are  equally 
probable  and  all  18  bits  of  memory  address  are  filly 
justified  from  the  start  to  permit  future  expansion  in 
a  clean  way.  (Buchholz  62  p. 130) 

The  analysis  apparently  did  not  include  an  estimate  of 

instruction  frequencies. 


Among  the  first  major  departures  from  fixed-length 
numerical  data  were  the  IBM  702  and  its  successors  (the  705 
and  1401) ,  and  the  IBM  1620.  On  the  1620,  an  integer  could 
be  two  or  more  decimal  digits,  each  digit  requiring  4  bits, 
but  instructions  were  fixed-length.  The  1401  allowed 
variable-length  data  and  instructions.  Although  these 
machines  were  immensely  popular  (Bell  &  Newell  71a  p.225) 
their  direct  influence  on  more  recent  machines  has  been 
quite  small. 


One  of  the  first  machines  to  decouple  the  addressing 
space  from  the  physically  available  main  memory  was 
Manchester  University's  ATLAS  computer.  Its  designers  chose 
to  group  program  sequences  and  data  aggregates  into  fixed- 
size  pieces,  called  "pages",  for  swapping  between  main  and 
secondary  memories.  For  the  same  purpose  the  Burroughs 
B5500  groups  program  and  data  into  variable-length 
"segments",  composed  at  compile-time  in  accordance  with  the 
program  structure.  In  support  of  the  latter,  McKeeman  has 
said : 


The  basic  question  underlying  this  aspect  of  the 
discussion  is  what  is  the  natural  program  segment  size? 
Analyses  of  scientific  programs  we  have  done  shows  a 
peak  in  number  of  segments  used  at  60  words  per 
segment.  However,  this  is  an  ill  defined  peak  lying 
between  20  and  100  words  per  segment,  and  even  at  that 
less  than  half  of  the  total  number  of  segments  used 
lies  in  this  range.  The  typical  system  designer  has 
chosen  to  impose  an  artificial  quantum  of  program  to 
address  by  selecting  a  page  size.  Several  problems  in 
the  programming  systems  immediately  arise  and  should  be 
noted  by  the  hardware  people. 

First,  in  selecting  this  unnatural  cleavage  plane  in  a 
program,  the  hardware  forces  the  system  programmer  into 
considerable  overhead  manipulation  to  handle  and  bridge 
the  arbitrary  cleavage  that  occurs  unpredictably  in  all 
programs  at  run  time.  If  the  programmer  ignores  these 
artificial  boundaries,  the  overhead  goes  up 
catastrophically. 

Second,  if  a  large  page  size  is  chosen  compared  to 
actual  segment  size  that  is  located  in  each  page  at  run 
time,  a  considerable  part  of  total  memory  is  wasted. 
It  is  an  accident  if  the  remainder  of  the  contents  of 
that  page  happen  to  be  needed  unless  the  programmer  has 
carefully  arranged  it. 

Third,  if  a  page  size  is  chosen  that  is  smaller  than 
the  actual  segment  to  be  run,  there  is  lost  time  as  the 
executive  program  handles  the  larger  number  of  page 
faults. 
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The  answer  is  to  use  a  varying  page  size  equal  ro  the 
actual  size  of  the  natural  program  segment  to  be  cun. 
This  approach  is  addressing  by  segment. 

A  comparative  measure  of  the  relative  value  of  these 
two  approaches  has  been  estimated  based  on  programs 
written  first  for  a  modern  page  addressed  machine  and 
second  for  a  contemporary  segment  addressed  machine. 
This  comparison  shows  that  for  equal  system  overhead 
time  measured  in  percent,  that  the  machine  using 
segment  addressing  uses  approximately  one-third  the 
memory  required  by  the  page  addressed  machine.  This 
represents  a  gain  of  factor  of  three  in  terms  of  main 
memory  utilization!  (McKeeman  67  p.416) 

Randell's  study  shows  a  similar  result: 

The  most  striking  result  is  the  apparently  general  rule 
that  rounding  up  requests  for  storage,  to  reduce  the 
number  of  different  sizes  of  blocks  coexisting  in 
storage,  causes  more  loss  of  storage  by  increased 
internal  fragmentation  [i.e.,  space  wasted  by  rounding 
up  the  requests]  than  is  saved  by  decreased  external 
fragmentation  [i.e.,  unusable  spaces  between  blocks]. 
(Randell  69) 

Barton,  who  was  a  major  contributor  to  the  B5500  and 

B6700  designs,  also  supported  segmentation  over  paging,  and 

extended  the  idea  to  all  storage  items. 

The  programmer  who  deals  directly  with  the  machine  has 
the  choice  of  fitting  the  data  representations  that  he 
uses  to  the  machine  and  thereby  either  restricting  his 
range  of  expressions,  wasting  storage,  or  using 
tailored  field  packing  and  unpacking  sequences,  thus 
wasting  both  the  storage  required  for  the  program  and 
increasing  execution  time. 

The  designer  of  a  conventional  machine  finds  the 
temptation  to  add  special  instructions  to  deal  with 
portions  of  words  almost  irresistable;  and,  in  fact, 
the  rapid  increase  in  number  of  instructions  after  the 
first  few  stored  program  machines  were  built  was  due  in 
part  to  this  and  in  part  to  the  necessity  to  fit 
compound  instructions  into  the  word  -  the  same  problem 
in  another  disguise.  With  separate  size  information 
the  instruction  list  can  be  shortened  and  simplified. 
(Barton  70  p.14) 
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I  believe  that  a  simple,  at  first  thought  rather 
trivial,  idea  can  open  the  way  to  the  solution  of  many 
programming  problems:  the  elimination  of  the  word  at 
machine  language  level.  (Barton  70  p.14) 

This  mysterious  quantity  known  as  the  "word”  has  been  of 

central  importance  in  the  design  of  most  computers,  and  is 

usua  iiy  one  of  the  first  victims  of  quantization.  Since,  by 

the  above  suggestion,  it  faces  elimination,  let  us  examine 

it  closely. 


2 . 2  The  Word 

Perhaps  the 

earliest 

use 

of  the 

term  in  a  computing 

context  was  in  a 

1946  paper 

by 

Burks , 

Goldstine  and  von 

Neumann  (Burks  et  al.  46) .  The  importance  they  attache!  to 
it  is  seen  in  their  statement  that  "the  moment  one  chooses  a 
given  component  as  the  elementary  memory  unit  (word) ,  one 
has  also  more  or  less  determined  upon  much  of  the  balance  of 
the  machine".  As  well  as  being  an  early  design  decision, 
the  word  size  is  generally  one  of  the  first  questions  asked 
about  a  new  computer-  Bell  and  Newell  consider  two  o£  the 
most  indicative  measures  of  computing  power  to  be  the  word 
size,  and  the  word  size  divided  by  the  memory  access  time 
(Bell  &  Newell  71a) . 

The  word  size  of  various  computers  has  ranged  fron  as 
small  as  4  bits  to  as  large  as  64  bits  and  can  be  correlated 
with  several  computer  features. 
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function:  scientific  computers  have  large  words  while 
business  computers  and  control  systems  have  snail 
words. 

number  of  data  types  and  operations:  multiple- word  data 

types  are  usually  provided  by  software  (notable 
exception:  double-word  numbers) ;  partial-word  data 
types  are  usually  provided  by  hardware.  So  in 
general,  computers  with  large  words  have  more 
hardware  data  types  and  more  operations. 

cost:  more  operations  require  more  hardware.  Data  paths 

are  often  one  word  wide,  meaning  still  more 
hardware  for  computers  with  large  words. 

In  spite  of  its  historical  importance,  the  definition 
of  "word”  is  not  at  all  clear.  The  front-running  candidates 
for  defining  word  size  are:  number  size,  instruction  size, 
register  size,  data  path  width  between  the  central  processor 
and  main  memory,  and  addressable  unit  in  main  memory.  In 
the  Burks,  Goldstine,  and  von  Neumann  machine  all  these 
quantities  were  equal  --  the  word  size  was  an  unambiguous  40 
bits.  Since  then,  machines  have  become  less  uniform. 
Figure  2  shows  these  quantities  for  several  machines,  aLong 
with  the  word  size  as  stated  by  the  manufacturer.  As  the 
chart  clearly  shows,  there  is  no  consensus  on  any  of  the 
candidates  for  word  size  definition.  Bell  and  Newell  appear 
to  have  chosen  the  data  path  width  between  main  memory  and 
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the  central  processor  as  their  definition  of  word  size  (not 
including  parity  bits  or  other  "special  purpose"  bits  such 
as  the  IBM  1401  F  bit  for  starting  and  ending  strings) . 
Actually,  they  define  word  size  as  n  bits,  and  speak  of  the 
"design  decision  that  the  unit  of  information  transfer 
between  components  will  be  a  word".  But  their  figures  give 
the  main  data  path  width  as  the  word  size,  even  when,  in  the 
manufacturer's  terminology,  it  is  not.  This  quantity  is 
fairly  well  defined  for  most  present-day  computers,  and  may 
be  worthy  of  the  importance  people  give  to  the  "word".  tfith 
current  technology  and  conventional  architecture,  this  data 
path  is  usually  a  major  system  bottleneck,  and  therefore  a 
major  factor  in  system  performance.  Systems  that  combine 
memories  to  take  advantage  of  the  speed  of  one  and  the 
economy  of  another  may  not  have  a  clear-cut  "main"  memory. 
If  the  data  paths  to  the  various  memories  differ  in  width, 
then  the  word  size  for  such  a  machine  must  be  defined  in 
some  other  way.  It  may  go  the  way  of  the  "byte",  which  once 
denoted  the  number  of  bits  to  be  operated  on  as  a  unit  by 
variable-field-length  instructions  (Bloch  59) ,  but  now 
usually  means  8  bits.  Or  it  may  disappear. 
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Figure  2:  The  Definition  of  the  "Word" 

There  is  no  consensus  on  any  of  the  candidates 

for  word  definition 

note:  these  sizes  do  not  include  parity 
or  other  "special  purpose"  bits 
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In  the  design  of  the  B1700  (Wilner  72a),  Barton's 
suggestion,  to  eliminate  the  word  at  the  machine-language 
level,  was  taken  in  all  of  its  meanings-  At  the  microcode 
level,  registers  and  the  main  data  path  are  known  to  be  24 
bits  wide,  and  the  addressable  unit  in  memory  is  1  bir.  But 
at  the  machine-language  (called  "S-language")  level, 
hardware  sizes  need  not  be  considered. 

2.3  Trade-offs  and  Balance 

It  is  sometimes  assumed  that  variable-length  operands 
must  be  operated  on  serially,  and  that  variable-lea gth 
operation  codes  require  serial  decoding,  resulting  in  a  slow 
machine-  To  quote  Wilner:  "Huffman  codes  may  require  much 
more  decoding  time,  since  bits  may  need  to  be  examined 
serially  until  the  length  of  the  field  manifests  itself,  but 
the  codes  can  minimize  storage-"  (Wilner  72b  p.581).  WiLner 
concludes  that  for  the  operation  codes  of  the  B1700  S- 
machine  for  the  SDL  source  language,  three  sizes  (4,  6  and 
10  bits)  yields  the  optimal  point  on  the  time-space  trade¬ 
off  curve-  However,  allowing  operation  code  lengths  to 
vary,  even  to  the  bit,  does  not  imply  that  decoding  must  be 
bit-serial-  There  is  a  third  factor:  hardware  cast; 
decoding  will  be  faster  if  the  decoder  is  wider,  up  to  the 
length  of  the  longest  code.  Similarly,  arithmetic  and 
logical  operations  need  not  be  performed  bit-serially .  The 
arithmetic  and  logical  unit  in  the  B1700  is  24  bits  wide. 
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and  may  be  used  iteratively  (for  long  operands)  and 
fractionally  (for  short  operands) .  Iterative  ani/or 
fractional  use  of  data  paths  has  been  standard  on  most 
computers  for  some  time.  The  Burroughs  "barrel  shifter" 
allows  the  data  path  between  main  memory  and  the  central 
processor  to  use  its  bandwidth  beginning  at  any  bit  in 
memory,  eliminating  any  need  for  alignment  or  justification. 
The  point  to  be  made  here  is  that  physically  fixing  the 
widths  of  the  instruction  decoders,  arithmetic  and  logical 
unit,  and  data  paths  imposes  no  constraints  on  the 
variability  of  instruction  and  data  lengths,  nor  are  there 
constraints  in  the  reverse  direction. 

If  computer  designers  take  advantage  of  the  freedoms 
expressed  in  the  above  paragraph,  they  will  solve  one  of 
their  greatest  dilemmas. 

Computer  design  is  still  driven  by  the  changes  in 
technology,  especially  in  the  varying  trade¬ 
off  s  ....  Thus  ,  our  designs  always  lag  the  technology. 
Those  that  are  reaching  for  the  new  technology  are 
extremely  crude.  Those  that  are  iterations  on  existing 
designs,  hence  more  polished,  fail  to  be  responsive  to 
the  newly  emerging  trade-offs.  (Bell  &  Newell  71b 
p.39  3) 

The  widths  of  hardware  components  should  be  chosen  to 
balance  their  speeds.  If  the  main  data  path  is  a  critical 
resource,  then  a  wide  decoder  is  wasted.  If  these  widths 
are  decoupled  from  information  widths,  as  we  are  suggesting 
they  should  be,  then  the  machine  may  take  advantage  of 
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changes  in  technology  and  still  maintain  the  balance  with  no 
consequences  at  the  machine-language  level- 

while  technology  determines  the  relative  widths,  the 
absolute  widths  are  chosen  by  weighing  static  costs 
(hardware)  against  running  costs  (speed) .  This  thesis  loes 
not  attack  that  question,  since  requirements  change  from 
installation  to  installation,  and  technology  changes  from 
year  to  year.  But  for  those  installations  where  the 
important  performance  measure  is  throughput,  and  not  average 
elapsed  time  per  job,  given  parallel  processing  (connected 
or  separate  processors) ,  the  important  figure  for  each 
processor  is  speed  per  hardware  cost,  and  this  figure  may  be 
high  for  bit-serial  machines. 

The  above  paragraphs  have  rejected  the  supposition  that 
the  amount  of  variability  in  instruction  code  lengths 
involves  us  in  a  time-space  trade-off.  We  now  make  rhe 
opposite  claim:  that  reducing  program  space  by  using 
variable-length  codes  will  reduce  execution  time.  To  quote 
Eastwood:  "The  chief  advantage  is  the  increased  effective 
bandwidth  in  transfer  of  compressed  program  information  from 
memory  to  the  processor  unit,  which  allows  higher  processor 
speed  for  a  given  memory  access  rate.”  (Eastwood  73).  This 
is  exactly  the  motivation,  expressed  earlier  by  Buchholz, 
for  changing  from  one  instruction  length  on  the  IBM  704  to 
two  on  the  IBM  7030  (Buchholz  62  p. 128) . 


Barton  *  s 
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suggestions  quoted  previously  came  from  his  concern  with 

space  and  programmability:  "The  reader  will  note  that  few 

of  the  preceding  remarks  stem  directly  from  any  emphasis  on 

speeds  of  processing.  Economy  of  representation  and  ease  of 

expression  are  the  motivations."  (Barton  70  p.15).  But  when 

his  suggestions  were  implemented,  the  emphasis  had  shifted: 

The  principal  reason  for  constructing  an  S-langnage 
which  is  different  from  that  dictated  by  the  Bl7D0's 
hardware  is  execution  speed.  It  is  possible  to  design 
machines  which  can  execute  programs  written  in  a  given 
language  more  than  10  times  faster  than  conventional 
general-purpose  computers.  Part  of  the  acceleration  is 
due  to  the  increased  appropriateness  of  the  machine* s 
fundamental  operations  and  part  is  due  to  increased 
program  compaction.  (Wilner  72c) 

We  base  the  claim  that  space  and  time  may  be  saved 
simultaneously  on  the  following  observation:  although 

static  and  dynamic  frequency  profiles  are  not  identical, 
they  are  similar  in  that  an  operation  or  operand  wnich 
appears  frequently  in  a  program  will  be  executed  or 
referenced  frequently,  and  vice  versa  (Alexander  72,  Wortman 
72)  .  Space  is  reduced  by  tailoring  the  length  of 
instructions  to  their  static  frequency  (frequency  of 
appearance  in  programs)  .  Time  is  reduced  by  tailoring 
lengths  to  dynamic  frequency  (execution  frequency) ,  making 
more  efficient  use  of  data  paths.  In  general,  then,  rhe 
optimum  for  space  should  be  close  to  the  optimum  for  time. 
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2.4  Experimental  Approach 


Early  machine  designers  were  satisfied  that  their 
simple  instruction  sets  could  compute  anything  they  waated. 
Only  when  machines  proved  they  could  work  reliably  did 
designers  start  to  worry  about  user  convenience.  "High- 
level"  languages  were  invented,  and  layers  of  software  were 
added  to  fill  the  gap  between  what  people  wanted,  and  what 
machines  offered.  One  of  the  most  successful  languages  was 
Fortran,  designed  to  make  the  IBM  704  more  attractive  to 
programmers.  As  languages  and  services  have  become  more 
people-oriented,  less  machine-dependent,  the  gap  has 
widened.  Yet  machine  designers  have  continued  to 
concentrate  on  improving  the  speed  of  instructions,  while 
their  systems  stumble  under  the  size  and  complexity  of 
software  necessary  to  compensate  for  badly  encoded  and 
inappropriate  instructions. 

...it  is  absurd  to  expect  carefully  engineered,  yery 
fast,  automatic  desk  calculators  [i.e.,  contemporary 
machines  ]  to  be  very  good  for  implementing  operating 
systems  or  compilers.  We  are  forced  to  manufacture  the 
operations  we  want  out  of  sequences  of  hardware 
operations  with  the  cbvious  result  that  the  programs 
become  large.  When  engineers  begin  to  seek  more 
efficient  encodings  for  commonly  used  sequences  of 
instructions,  progress  toward  the  modern  computer  may 
begin.  (McKeeman  67  p.414) 

Language-directed  machine  design  refers  to  the  effort  to 
make  machine  instructions  and  structures  more  appropriate 
for  representing  and  executing  programs  written  in  high- 
level  languages.  (For  extensive  bibliographies,  see  Abrams 
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70  and  Wortman  72.)  These  efforts,  for  the  most  part, 
relied  on  the  intuition  and  experience  of  indiviiual 
designers.  Wortman  based  the  design  of  his  machine  an  a 
statistical  analysis  of  the  source  language  for  which  it  was 
intended.  His  primary  concern  was  choosing  an  instruction 
set  whose  semantics  matched  the  requirements  of  his  saaple 
source  programs,  rather  than  matching  the  representation  of 
instructions  and  data  to  their  information  content. 

Following  the  language-directed  approach,  we  shall 
assume  that  all  instructions  are  generated  automatically 
from  high-level  source  programs.  This  has  two  consequences: 
only  those  sequences  of  actions  representable  in  the  source 
language (s)  need  representations  in  machine-language,  ani  we 
shall  not  worry  about  making  the  machine  instructions 
readable  by  a  programmer. 

Statistics  on  the  use  of  high-level  languages  have  neen 
gathered  by  several  people.  Wichmann  used  the  frequencies  of 
Algol  operations  and  statement  types  in  his  comparative 
evaluation  of  several  computer  systems  (Wichmann  73) ;  Knuth 
looked  at  Fortran  program  profiles  as  an  aid  to  compiler 
writing,  particularly  optimization  (Knuth  70) ;  Alexander 
gathered  similar  statistics  from  XPL  programs  (Alexander 
72);  Wortman* s  statistics  on  Student  PL  programs  were  a  part 
of  his  machine  design  procedure  (Wortman  72) .  Less  is  known 
about  the  values  of  variables.  Sweeney  examined  the 


-23- 


distribution  of  exponents  of  floating-point  numbers  (Sweeney 
65) ,  and  Hamming  gave  theoretical  reasons  for  the 
logarithmic  decline  in  the  leading  digit  of  the  mantissas  of 
naturally  occurring  numbers  in  floating-point  notation 
(Hamming  70,  Knuth  69  pp.  218-228).  We  shall  draw  freely 
from  these  statistics  whenever  appropriate. 

The  Burroughs  B1700  was  designed  to  accommodate  machine 
instructions  and  data  having  variable-length,  frequency- 
based  encodings  (Wilner  72af>b)  •  It  does  not  have  a  fixed 
language  at  the  level  of  other  computers'  machine-languages, 
but  rather  it  interprets  supplied  and  user-defined  machine- 
level  languages  at  the  micro- program  level.  At  this  level, 
memory  is  bit-addressable,  all  hardware  (data  paths, 
registers,  adder,  etc.)  may  be  used  partially,  and  serially 
re-used.  Because  of  these  features,  many  of  the  ideas 
presented  in  this  thesis  could  easily  be  implemented  on  the 
B 1 700 . 

2 . 5  Objective 

Our  general  purpose  is  to  design  the  best  machine  for  a 
given  environment.  Specifically,  if  our  environment  is 
represented  by  a  class  of  programs  to  be  run  on  the  machine, 
we  wish  to  minimize  the  length  of  the  machine-language 
representation  of  the  class,  or  equivalently,  the  average 
length  of  programs  (weighted  by  frequency  of  program)  in  the 
class.  The  entropy  of  the  class  gives  us  an  almost 
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achievable  target  to  aim  for,  and  redundancy  gives  us  an 
absolute  measure  of  performance  for  any  machine  in  the  given 
environment.  Section  3  gives  definitions  of,  and  methods 
for  determining,  the  entropy,  information  content,  and 
redundancy  of  programs. 

One  may  object  to  the  goal  of  minimizing  redundant/  on 
the  grounds  that  it  is  needed  for  reliability.  h  macaine 
instruction  may  be  in  error  for  two  reasons:  either  a  bit 
which  was  set  correctly  has  spontaneously  changed,  or  the 
instruction  as  originally  formed  was  incorrect.  Some  forms 
of  redundancy  in  the  machine  language  allow  us  to  detect 
some  of  these  errors.  If  there  are  unused  operation  codes, 
or  addresses  which  are  illegal  or  inaccessible,  and  if  the 
error  happens  to  result  in  one  of  these  illegal 
instructions,  it  can  be  detected  and  the  offending 
instruction  identified.  If  the  error  results  in  a  legal 
instruction,  it  may  be  detected  indirectly  but  escape 
identification,  or  it  may  escape  detection.  Other  forms  of 
redundancy,  such  as  inter-instruction  dependencies,  do  not 
provide  any  error  detection  ability. 

The  use  of  accidental  redundancy  in  machine-language 
instructions  for  error  detection  is  at  best  a  haphazard 
approach,  and  at  worst  a  poor  excuse  for  badly  designed 
codes.  The  purpose  of  machine- language  is  to  specify  a 
sequence  of  actions  as  succinctly  as  possible.  Error 
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detection  ability  is  important  enough  to  deserve  its  own 
separate  mechanisms,  specially  designed  for  that  purpose, 
such  as  parity  bits  or  tag  bits  (B6700,  Iliffe  68,  Feustal 
72).  Error-correcting  codes  can  ensure  the  validity  of  bit 
values  up  to  any  desired  error  density;  tag  bits  can  ensure 
that  no  data  is  executed,  or  its  type  mistaken,  although  we 
may  prefer  to  place  such  debugging  aids  in  the  software. 
The  issues  of  machine- language  representation  and  srror 
detection  are  separable,  and  our  concern  is  with  the  former. 
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3  ENTROPY  AND  INFORMATION  CONTENT  OF  PROGRAMS 
3 • 1  Definitions 

In  Information  Theory  (Shannon  49,  Ash  65)  the  entropy 
of  a  set  of  n  messages  is  defined  as 
H  =  -SUM  Pi  log  Pi 

where  Pi  is  the  probability  or  relative  frequency  of  rhe  ith 

message.  Pi  >  0,  SUM  Pi  =  1.  (In  this  section,  all 

logarithms  are  to  the  base  2,  and  all  sums  are  from  1  ta  n. 
The  notation  ” SUM ”  is  used,  for  typographic  reasons,  in 
place  of  the  more  usual  capital  sigma.  We  may  allow  Pi  =  0 
by  taking  0  log  0=0,  although  by  a  suitable  adjustment  of 
the  message  set,  this  is  unnecessary.)  The  entropy  of  a  set 
of  messages  represents  the  average  amount  of  choice  involved 
in  sending  a  message,  or  the  amount  of  uncertainty  on  the 
part  of  the  receiver  as  to  which  message  will  be  sent. 
Entropy  is  independent  of  any  representation  of  the 

messages.  It  is  a  close  (within  1  bit)  lower  bound  on  the 
average  length  of  messages  (weighted  by  frequency  of 

message),  according  to  a  "best"  binary  representation,  i.  e. , 
a  binary  representation  that  minimizes  the  average  lengta  of 
a  message. 

We  now  define  the  information  content  of  a  program  (or 
message)  in  a  class  (or  set)  of  programs  (or  messages)  as 
I  (PROGRAMi)  =  -log  Pi 
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It  is  the  amount  of  information,  measured  in  bits,  that 
distinguishes  it  from  the  other  members  of  the  class.  Like 
entropy,  the  information  content  of  a  program  is  indepenient 
of  the  representation,  and  is  a  close  (within  1  bit) 
estimate  of  the  number  of  bits  taken  by  a  best  binary 
representation.  Entropy  is  then  the  average  information 
content  of  a  program. 

In  Information  Theory,  the  redundancy  of  a  set  of 
messages  is  defined  as 

D  =  1  -  (relative  entropy) 

where  the  relative  entropy  is  the  actual  entropy  divide!  by 
the  maximum  possible  entropy  of  the  set.  For  a  set  of  n 
messages,  entropy  is  at  its  maximum  when  Pi  =  1/n  for  each 
i.  Redundancy  is  therefore  1  -  H/log  n.  This  definition  is 
independent  of  representation,  and  unsuitable  for  our 
purposes.  We  shall  define  the  redundancy  of  a 
representation  as 

D*  =  1  -  H/(average  message  length) 
where  the  length  of  a  message  depends  on  its  representation. 
With  this  definition,  redundancy  measures  the  proportion  of 
the  representation  that  exceeds  the  entropy. 

3.2  Determination  of  Entropy 

If  we  are  given  the  entire  class  of  programs  to  be  run 
on  our  machine,  and  the  probability  or  frequency  of  each 
program,  then  calculating  the  information  content  of  each 
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program  and  the  entropy  of  the  class  is  straightforward. 
More  than  that,  an  algorithm  for  determining  a  oast 
representation  has  been  given  by  Huffman  (Huffman  52) .  But 
few  machine  designers  are  given  the  luxury  of  such 
information. 

Normally  we  know  only  that  the  programs  to  be  run  on 
our  machine  come  from  an  infinite  class,  such  as  the  set  of 
all  Algol  programs,  or  the  set  of  all  PL/I  programs,  or  the 
union  of  such  sets.  The  definition  of  entropy  can  easily  be 
extended  to  cover  an  infinite  class,  and,  if  we  are  given 
the  probability  distribution,  we  can  still  compute  our 
target  for  the  length  of  the  average  program.  According  to 
the  classical  definition,  the  redundancy  of  any  infinite 
class  with  finite  probabilities  is  100%,  but  the  redundancy 
of  representation  provides  a  performance  measure  even  for 
infinite  classes.  If  we  are  lucky,  we  may  find  a  good 
machine-language  representation  for  the  particular 
distribution,  but  we  know  of  no  algorithm  for  determining  a 
best  representation  for  an  arbitrary  distribution  over  an 
infinite  class. 

Normally  we  are  not  given  a  probability  distribution. 
We  are  given  a  small  sample  of  programs  from  which  we  must 
derive  a  reasonable  estimate  of  the  distribution  for  the 
class.  The  sample  is  intended  to  be  representative  to  some 
degree  of  the  entire  class.  The  answer  to  the  question 
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"Just  how  representative?”  will  provide  the  methol  of 
determining  the  class  entropy. 

For  a  large  enough  sample  we  may  assume  that  the 
relative  frequencies  of  the  language  tokens  in  the  sample 
are  representative  of  the  frequencies  in  the  class. 
Similarly  we  may  assume  that  the  relative  frequencies  of 
production  numbers  in  a  parse  of  the  sample,  or  that  the 
frequencies  of  states  in  the  parser,  represent  their 
frequencies  over  the  entire  class.  Since  a  token  sequence 
is  a  particular  representation  of  programs,  the  entropy  of 
the  token  set  (information  content  of  an  average  token) 
multiplied  by  the  number  of  tokens  in  an  average  program 
(over  the  sample)  will  be  an  estimate  of  the  entropy  of  the 
class  of  programs.  Similarly  the  production  sequence  and 
parser  state  sequence  provide  estimates  of  the  entropy. 
Unfortunately  these  estimates  may  be  poor,  and  they  depend 
rather  heavily  on  the  choice  of  representation. 

Almost  as  certainly,  the  frequencies  of  token  pairs, 
production  pairs  or  state  pairs,  are  representative.  tfith 
decreasing  confidence,  we  may  assume  that  higher  m-tuple 
frequencies  are  representative.  In  the  following 
paragraphs,  j  will  stand  for  an  (m-1) -tuple  of  subscripts. 
If  Pij  is  the  probability  of  the  ijth  m-tuple  of  symbols 
from  an  alphabet  of  n  symbols  (tokens,  productions  ere.), 
and  N  is  the  length  in  symbols  of  an  average  program,  then 


-30- 


the  estimate  of  entropy  based  on  non-overlapping  m-tuple 
frequencies  is 

G  (m)  =  -  (N/m)  SUM  Pij  log  Pij 

ij 

As  m  increases,  these  estimates  form  a  monotonically 
decreasing  sequence  (Appendix  A. 1)  .  Furthermore,  as  m 
becomes  large,  m-tuples  become  entire  programs,  and  the 
sequence  of  estimates  converges  to  the  entropy  of  the 
sample.  If  we  stop  too  soon,  we  will  not  be  taking  enough 
advantage  of  the  sample,  and  our  estimate  will  be  too  high. 
If  we  stop  too  late,  we  will  be  taking  too  much  advantage, 
and  our  estimate  will  be  low  (unless  the  sample  is  the 
complete  class,  in  which  case  the  sample  entropy  is  the 
class  entropy).; 

More  effective  use  can  be  made  of  m-tuple  frequenoies 
by  considering  the  conditional  entropy  of  the  next  symbol 
when  the  m-1  preceding  symbols  are  known.  The  mth  estimate 
of  class  entropy  based  on  conditional  symbol  entropy  is 

F  (m)  =  -N  SUM  Pij  SUM  Pijk  log  Pi  jk 
ij  k  Pij  Pij 

where  now  the  frequencies  include  overlapping  tuples.  Like 
G (m) ,  F(m)  is  a  monotonically  decreasing  function  of  m 
(Appendix  A. 2),  converging  to  the  entropy  of  the  sample;  and 
F  (m)  <  G  (m)  (Appendix  A.  3)  . 

The  problem  is  to  decide  which  m  makes  F  (m)  or  G  (m)  the 
best  estimate  of  H.  He  want  m  to  be  large  enough  to  include 
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all  dependencies  or  statistical  influences  characteristic  of 
the  class  and  none  that  are  due  only  to  the  finite  size  of 
the  sample-  We  find  this  m  by  formulating  an  appropriate 
hypothesis,  and  applying  a  statistical  test  to  determine,  to 
a  given  level  of  confidence,  whether  our  sample  is 
sufficient  to  support  (or  reject)  the  hypothesis.  Suppose 
the  sample  were  the  complete  class.  Now  G(m*1)  =  3(m) 
implies  G  (m)  =  G(m-1)  (Appendix  A.1).  For  most  programming 
languages,  pairs  of  tokens  (productions,  states)  are  not 
independent  (usually  not  all  pairs  are  legal)  ,  hence  by 
Appendix  A.1,  G(2)  *  G(1)  and  therefore  G  (m)  *  H  for  an/  m. 
The  F  estimates  are  more  cooperative;  F(m)  may  converge  to  H 
for  some  finite  m.  Of  course,  when  it  does,  F(m+1)  =  F(m) 
ever  after.  Now  F(m+1)  =  F  (m)  if  and  only  if  Pijk  = 
Pij*Pjk/Pj  (Appendix  A. 2).  With  our  limited  sample,  we  wish 
to  find  the  last  m  for  which  we  can  confidently  reject  the 
hypothesis  Pijk  =  Pi j*P jk/P j,  i.e.,  the  largest  distance 
over  which  the  sample  indicates  that  a  dependency  exists. 
With  limited  computer  resources,  one  may  wish  to  make  an 
additional  assumption:  that  a  dependency  of  order  m  implies 
a  dependency  of  order  m-1.  This  allows  one  to  stop  at  the 
first  m  for  which  a  dependency  is  not  indicated. 

We  have  discussed  one  method  of  ensuring  that  we  do  not 
over-use  the  sample:  by  limiting  our  knowledge,  at  any 
point  in  the  sequence  of  symbols  representing  the  sample,  to 
the  m  preceding  symbols.  A  different  kind  of  limit  on  our 
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use  of  the  sample  is  the  following:  instead  of  calculating 
the  entropy  of  each  symbol  conditionally  upon  the  complete 
information  contained  in  a  limited  number  of  preceding 
symbols,  we  may  calculate  the  entropy  of  each  symbol 
conditionally  upon  part  of  the  information  contained  in  the 
complete  preceding  symbol  sequence.  Suppose  that  the 
language  under  consideration  is  describable  by  BNF,  and  may 
be  parsed  according  to  the  usual  stack  methods.  During  the 
parse,  the  parse  stack  is  not  a  complete  representation  of 
what  has  been  seen  so  far.  It  retains  exactly  the 
information  that  determines  what  may  legally  be  seen  in  the 
remainder  of  the  program;  therefore  limiting  ourselves  to 
the  information  contained  in  the  parse  stack  at  each  step  in 
the  token  (or  production)  sequence  seems  to  be  a  reasonable 
constraint  on  the  use  of  the  sample. 

Although  the  decision  to  confine  ourselves  to  the 
parse-stack  information  is  somewhat  arbitrary,  as  is  the 
choice  of  confidence  level  for  a  statistical  test  of 
independence,  it  should  be  emphasized  that,  in  the  end,  any 
answer  to  the  question  "How  representative  is  the  sample?”, 
and  therefore  any  estimate  of  entropy,  is  subject  to  error. 
As  we  said  in  the  previous  section,  an  estimate  of  entropy 
serves  as  a  goal  in  the  design  and  representation  of  a 
machine-language;  but  more  important,  the  above  methods  for 
determining  entropy  will  suggest,  in  Section  4.4,  practical 


methods  for  improving  a  machine-language 
representation. 


■ 
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4  instkuciions 

In  this  section  we  shall  identify  common  dependencies 
and  contributors  to  redundancy  in  machine-language 
instructions,  and  develop  techniques  for  removing  them. 
Before  we  can  discuss  encodings  of  instructions,  we  oust 
first  ask  "What  is  an  instruction?".  Towards  one  extreme, 
each  program  can  be  regarded  as  a  single  instruction.  Then, 
by  knowing  the  frequency  of  each  program  (instruction!  in 
the  class  of  programs  to  be  run  on  our  machine,  and  assuming 
independence  of  programs,  we  could  choose  an  instruction 
encoding  that  would  minimize  the  average  space  occupied  oy  a 
program  (Huffman  52).  This  may  be  feasible  for  a  very  snail 
class  of  very  simple  programs,  run  frequently  enough  that 
frequencies  are  meaningful.  But  for  most  environments  it  is 
not. 


The  smaller  the  unit  of  action  that  an  instruction 
represents,  the  easier  the  instruction  is  for  people  to 
understand,  and  the  less  expensive  is  the  hardware  needel  to 
perform  it.  However  the  space  required  by  an  average 
program  will  tend  to  increase.  This  occurs  for  two  reasons. 
First,  assuming  the  frequency  (or  probability)  of  each 
instruction  remains  independent  of  the  surrounling 
instructions,  the  information  content  of  an  average  program 
remains  constant,  but  the  redundancy  in  the  minimum 
redundancy  encoding,  limited  to  one  bit  per  instruction. 
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becomes  a  larger  proportion  of  each  instruction  (unless  it 
was  very  nearly  zero,  in  which  case  it  may  be  unchanged) . 
Second,  at  some  stage  the  independence  assumption  breaks 
down.  After  that,  as  the  unit  of  action  becomes  smaller, 
the  dependence  of  one  action  upon  other  actions  increases, 
and  so  does  the  space  required  by  an  average  program 
composed  of  such  instructions.  This  trade-off,  between 
under standability  and  hardware  expense  on  the  one  hand,  and 
program  space  on  the  other,  should  determine  the  amount  of 
action  to  be  represented  by  an  instruction. 

On  different  machines  instructions  have  taken  many 
different  forms.  The  number  of  operations  has  varied 
between  one  (van  der  Poel  56)  to  several  hundred  (aost  large 
modern  machines) .  The  number  of  data  addresses  per 
instruction  has  varied  from  three  (two  sources  and  a 
destination,  e.g.,  Honeywell  8200)  to  zero  (all  operands 
implicit,  e.g.,  KDF-9,  B5500) .  Some  machines  have  a  branch 
address  in  each  instruction  to  control  program  flow  (IBM 
650,  Bendix  G15  and  most  micro-code);  others  have  a  separate 
branch  instruction  (IBM  360  and  many  others) .  Some  machines 
have  an  index  field  within  an  instruction  to  modify  a  data 
address  (IBM  360  and  many  others) ;  others  have  an  iadex 
instruction  to  modify  a  data  address  in  the  following 
instruction  (B6700  and  Myriad) .  Operand  formats  may  be 
stated  explicitly  in  separate  fields  (PDP-11  "addressing 
mode"  or  PDP-1,  PDP-8  indirect  bit)  or  they  may  be  imolied 
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by  the  operation  code.  To  some  extent  these  differea 
give  the  various  mach ine- languages  differ 

characteristics:  manufacturers  of  machines  with  a  la 

operation  repertoire  claim  "power”  and  "flexibility",  wi 
manufacturers  of  machines  with  a  small  operation  reperto 
(usually  mini-computers)  claim  compactness;  similarly 
many-address  machines  claim  flexibility  while  the  no-adic 
(stack)  machines  claim  compactness.  But  to  some  ext 
these  differences  are  just  terminology:  a  field  ia 
instruction  may  be  given  a  separate  name,  or  considered 
nameless  subfield  of  the  operation  code. 

We  shall  distinguish  four  parts  of  an  instruction: 

1.  constant  data  (literals) 

2.  branch  (instruction)  addresses 

3.  data  addresses,  and 

4.  the  rest,  including  operation  code 

specification  of  the  form  of  operand  (s). 

The  first  three  are  potentially  infinite  classes;  the  1 
is  a  small  finite  class.  The  number  of  useful  operat 
codes,  that  is,  operation  codes  emitted  by  a  compiler, 
often  small  (IBM  360  XPL:  40  (McKeeman  et  al.  70) ;  IBM  7 
Fortran:  19  (IBM  7090)).  For  this  reason,  algorithmic  c 

compaction  technigues  that  are  suitable  for  operation  co 
are  not  suitable  for  the  first  three  classes.  These  wilL 
discussed  under  separate  sections  below. 
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4.1  Constant  Data 

As  the  Introduction  suggests,  knowledge  of  the 
distribution  of  constants  that,  appear  in  programs  can  Lead 
to  an  improved  representation  for  those  constants.  The 
three  common  data  types  are  logical  or  boolean  valaes, 
character  strings,  and  numbers.  These  types  come  in  two 
forms:  simple,  unstructured  values,  or  structured  aggregat.es 
of  values.  We  shall  consider  unstructured  constants  first. 

Since  there  are  only  two  boolean  values,  true  and 
false,  the  appropriate  representation,  a  single  bit,  is 
obvious,  and  requires  no  further  discussion. 

Large  data  banks  are  often  composed  primarily  or 
completely  of  character  strings,  and  the  possibility  of 
large  savings  in  space  and  transmission  time  have  motivated 
studies  of  text  compression  (Weaver  74,  Hagamen  et  al.  72, 
Snyderman  Z  Hunt  70) .  A  standard  technique  is  to  employ 
unused  bit  patterns  in  a  fixed-length  code  to  represent 
common  pairs  of  characters.  We  discuss  this  technique,  and 
others  using  variable-length  codes,  as  they  apply  to  op¬ 
codes  (see  Section  4.4),  and  the  results  stated  there  are 
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character  strings  have  received  so  much  attention  in  the 
literature,  and  since  better  methods  of  compressing 
sequences  of  symbols  from  a  finite  alphabet  are  discissed 
elsewhere  in  this  thesis,  we  shall  not  pursue  further  the 
merits  of  various  standard  or  non-standard  character 
representations.  We  shall,  however,  list  three  methods  of 
delimiting  the  string: 

(a)  One  bit  in  the  representation  of  each  character 
tells  whether  it  is  the  final  character  in  the 
string.  The  IBM  1401  uses  this  method. 

(b)  One  bit  pattern,  not  used  to  represent  any 
character,  is  used  to  indicate  the  eni  of  the 
string.  The  ASCII  and  EBCDIC  communication  oodes 
and  the  IBM  702  use  this  method. 

(c)  The  length  of  the  string  is  given  explicitly, 
either  at  the  head  of  the  string,  or  prior  to  it 
on  the  access  path.  The  B6700  and  IBM  360  use  this 
method. 

Which  method  of  the  three  is  best  depends  on  character 
representation,  and  on  the  distribution  of  string  lengths. 
For  a  fixed-length  character  representation,  method  (a)  fill 
be  better  than  method  (b)  if  the  average  number  of 
characters  in  a  string  is  less  than  the  number  of  bits  in  a 
character,  and  worse  if  greater.  How  method  (c)  compares 
with  methods  (a)  and  (b)  depends  on  how  integers  are 
represented,  which  we  shall  now  discuss. 
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Numbers  in  computing  come  in  several  varieties,  e.g.: 

1.  integers 

2.  fixed-point  numbers 

3.  floating-point  numbers 

4.  rational  numbers 

5.  complex  rational  numbers 

6.  number  ranges  (intervals) 

The  last  four  varieties  may  be  represented  by  pairs  or 
tuples  of  fixed-point  numbers,  and  fixed-point  numbers  are 
scaled  integers.  Of  the  six,  integers  are  the  most  conmon 
form  of  constant,  even  in  scientific  applications,  where 
they  are  used  as  array  indices  and  loop  controllars. 
Accordingly,  we  shall  consider  integers  first. 

Some  work  has  already  been  done  to  find  the 
distribution  of  integer  constants  in  samples  of  Student  PL 
and  XPL  programs  (Wortman  72,  Alexander  72) .  We  present 
below  a  variety  of  number  schemes  with  enough  parameters  to 
match  a  variety  of  distributions,  which  therefore  reluce 
space  and  eliminate  overflow,  and  which  do  not  result  in  an 
obscure  representation  or  one  that  complicates  operations 
that  use  these  constants. 

Positional  number  systems  represent  a  number  as  a 
sequence  of  digits.  The  three  methods  listed  above  for 
delimiting  character  strings  are  also  applicable  to  a 
sequence  of  digits: 
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(a)  One  bit  in  the  representation  of  each  digit  (or 
between  digits)  tells  whether  the  current  (or 
previous)  digit  is  the  final  one  in  the  nunber. 
We  call  this  bit  the  '’go-stop’'  bit. 

(b)  One  bit  pattern,  not  used  to  represent  any  digit, 
is  used  to  indicate  the  end  of  the  number.  He 
call  this  pattern  the  "stopper". 

(c)  The  length  of  the  number  is  given  explicitly, 
e.g-,  using  one  of  methods  (a),  (b)  or  (c)  , 
followed  by  the  digits  of  the  number. 

We  may  consider  the  semi-infinite  sequence  of  digits 

...  Di  ...  D3  D2  D1  DO 
to  represent  the  integer 
inf 

n  =  SUM  Di*(B**i) 
i=0 

where  B  is  an  integer  greater  than  1,  each  digit  Di 
represents  an  integer  in  the  range  0  <  Di  <  B,  and,  for  all 
i  >  log  (n)  (base  B) ,  Di  =  0.  The  complement  form  for 
negative  integers  may  be  defined  by  the  algorithms  for 
arithmetic.  For  example,  in  base  4, 

• • • 3. . . 3333 

+  .  .  .  0 _ 0001 

=  . .  .0. . . 0000 

Therefore  ...3.. .3333  represents  -1. 

The  uniform  number  schemes  are  a  generalization  of  this 
form  of  complement  arithmetic.  The  base  B  may  be  any  aon- 
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zero  integer  (positive  or  negative) ,  and  the  digits  may 
represent  any  |B|  integers  which  are  different  modulo  B. 
Digits  representing  more  than  |B|  integers,  including  | B| 
which  are  different  modulo  B,  may  be  used,  but  then  the 
number  representations  will  not  be  unique.  The  name 
"uniform”  comes  from  the  fact  that  the  successor  algorithm 
is  uniform  throughout  the  scheme:  the  successor  to  any 
number,  negative,  zero  or  positive,  is  obtained  in  precisely 
the  same  manner,  without  first  ascertaining  the  sign.  The 
addition,  subtraction,  multiplication  and  division 
algorithms  for  any  uniform  number  scheme  are  the  same  as  for 
complement  arithmetic.  For  different  digit  sets,  the  carry 
properties  may  be  different,  affecting  the  design  and 
complexity  of  hardware  to  perform  arithmetic. 

Appendix  B. 1  lists,  as  examples,  some  members  of  the 
base  four  family  of  uniform  number  schemes.  In  each  uniform 
number  scheme,  there  are  only  a  few  different,  semi¬ 
infinite,  repetitive,  initial  portions  of  the  digit 
sequences,  so  they  can  easily  be  encoded  in  a  few  bits.  In 
Appendix  B. 1  we  have  indicated  the  repeating  unit  by 
underscoring  it.  For  example,  in  four's  complement,  each 
non-negative  number  begins  with  a  string  of  Os,  and  each 
negative  number  begins  with  a  string  of  3s.  The  obvious 
representation  gives  the  initial  string  a  single  bit,  and 
each  digit  thereafter  two  bits,  with  go-stop  bits 
interspersed.  We  can  improve  by  noticing  that  the  digit 
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following  initial  Os  cannot  be  0f  and  the  digit  following 
initial  3s  cannot  be  3.  The  numbers  in  parentheses  are  the 
lengths  in  bits  of  the  representation  which  incorporates  the 
above  improvement.  It  can  be  seen  that  this  representation 
of  four's  complement  favours  negative  numbers  slightly,  in 
that  some  negative  numbers  have  shorter  representations  than 
the  corresponding  positive  numbers.  In  a  fixed-length 
coding  scheme  this  bias  shows  up  in  the  ability  to  represent 
one  more  negative  number  than  positive.  Different  members 
of  a  family  of  uniform  number  schemes  are  biased 
differently,  as  Appendix  E. 1  shows. 

A  variable-length  sign-and-magnitude  representation  may 
be  obtained  from  any  uniform  number  scheme  by  ignoring  the 
representations  of  the  negative  numbers  and  appending  a  sign 
to  the  representations  of  the  positive  numbers. 

If  digits  are  represented  by  a  fixed-length  rode,  then 
base  B  =  2**(b-1)  is  appropriate  with  delimiting  method  (a), 
using  one  bit  in  every  b  bits  to  say  stop  or  keep  going;  and 
base  B  =  (2**b) -1  is  appropriate  with  delimiting  method  (b) , 
reserving  one  pattern  for  a  stopper.  When  B  =  b  =  1, 
methods  (a)  and  (b)  become  indistinguishable,  and  the 
resulting  code  is  called  a  unary  number  scheme.  As  an 
example  of  delimiting  method  (c) ,  the  number  scheme  in 
Appendix  B. 2  represents  the  length  in  unary,  and  integer 
part  in  binary  sign-and-magnitude.  Due  to  its  importance. 
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we  have  placed  the  sign  at  the  beginning.  This  schema  is 
positively  biased,  i.e.,  it  matches  a  distribution  in  which 
positive  integers  are  more  common  than  negative  integers. 

Our  object  in  this  section  has  been  to  present  a 
variety  of  variable-length  number  representations  that  are 
amenable  to  arithmetic  operations,  from  which  a  machine 
designer  can  choose  one  that  fits  the  number  distribution  in 
his  machine’s  environment.  By  selecting  a  larger  base,  he 
can  match  a  spread  distribution;  a  small  base  matches  a 
peaked  distribution.  To  a  lesser  extent,  the  stopper  method 
of  delimiting  length  matches  a  spread  distribution,  and  the 
go-stop  method  matches  a  more  peaked  distribution.  Having 
chosen  a  base  B,  the  machine  designer  can  choose  those  | B\ 
digits,  different  modulo  B,  that  result  in  a  representation 
which  best  matches  the  bias  or  skew  of  his  number 
distribution.  For  a  given  set  of  digits,  he  can  affect  the 
bias  by  selecting  an  appropriate  variable-length  encoding  of 
the  initial,  semi-infinite  portion  of  his  numbers.  It  he 
really  wants  to  do  some  fine  tuning  at  the  expense  of  some 
circuitry,  he  may  choose  the  representation  of  the  initial 
portion  to  account  for  the  fact  that  the  probability  that 
the  leading  digit  is  d  decreases  logarithmically  as  d 
increases.  A  discussion  and  derivation  of  the  distribution 
of  leading  digits  may  be  found  in  Knuth  (Knuth  69  pp.  218- 
228)  . 
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As  we  mentioned  previously,  more  complicated  varieties 
of  numbers,  such  as  floating-point  and  rational  numbers,  may 
be  represented  simply  by  pairs  or  tuples  of  integers.  It  we 
wish  to  include  one  or  more  of  these  data  types  in  our 
machine,  we  may  do  so  in  either  of  two  ways:  we  may  add  new 
operations  to  operate  on  the  new  data  type,  or  we  may 
represent  all  numbers  as  the  more  complicated  type.  This 
latter  suggestion  becomes  very  attractive  when  only  one  or 
two  extra  bits  are  required  for  a  zero  exponent  or  unit 
denominator. 

A  structured  constant  is  an  aggregate  of  structured  or 
unstructured  constants.  Without  loss  of  generality  we  may 
remove  the  recursion  from  this  definition,  and  represent  the 
aggregate  as  a  linear  array  of  unstructured  constants,  since 
the  appearance  of  a  more  complex  structure  can  be  achieved 
by  performing  an  arithmetic  transformation  on  the  iadex. 
The  indexing  operation  requires  either  that  we  decipher  all 
elements  from  the  beginning  of  the  array  until  we  find  the 
one  we  want,  which  is  intolerably  slow,  or  that  all  elements 
be  given  the  same  space.  For  structured  constants,  we  ceuld 
give  each  element  the  maximum  space  needed  by  any  element, 
but  this  would  make  establishing  the  array  difficult,  and 
waste  space  in  arrays  whose  elements  ha’ve  quite  different 
lengths.  Alternatively,  we  may  provide  each  element  with  a 
standard  size  space  for  the  data  type,  and  provide  an 
indirection  mechanism  for  those  elements  that  do  not  fit  the 
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standard  space.  Considerations  concerning  the  best  standard 
space  and  best  indirection  mechanism  are  similar  to 
considerations  for  variables,  discussed  in  Section  5.  We 
shall  therefore  subsume  discussion  of  structured  constants 
under  the  discussion  of  variables. 

4 . 2  Branch  Addresses 

For  each  of  the  other  parts  of  an  instruction  (op¬ 
codes,  constants  and  data  addresses)  the  class  of  objects  to 
be  represented  is  unaffected  by  the  choice  of 
representation.  Therefore  the  more  variability  allowed  in 
the  lengths  of  their  representations,  the  more  space  can  be 
saved.  But  for  branch  addresses,  the  class  is  affected: 
allowing  instruction  lengths  to  vary  to  a  finer  unit  of 
storage  increases  the  class  of  branch  addresses,  and 
therefore  costs  bits  on  each  branch  address.  On  the  other 
hand,  variability  in  branch  address  lengths,  as  in  other 
field  lengths,  allows  us  to  match  the  distribution  better, 
and  save  space.  This  consideration  determines  the  optimum 
amount  of  variability  in  instruction  lengths. 

Let  the  length  of  each  field  in  an  instruction,  and 
therefore  the  length  of  the  instruction,  be  a  multiple  oE  b 
bits.  Let  the  average  number  of  (separately  encoded)  fields 
in  an  instruction  be  f  (typically  f  =  2  to  4) .  Now,  the 
redundancy  in  a  minimum  redundancy  encoding  is  limited  to 
one  character  =  b  bits  per  field;  if  the  actual  redundancy 
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is  proportional  to  the  maximum  possible  redundancy,  with 
proportionality  factor  w,  then  the  wasted  space  in  an 
average  instruction  is  f*b*w  bits  (typically  0.1  <  w  <  D.5). 
Changing  the  unit  of  variability  from  2*b  bits  to  b  bits 
changes  the  waste  per  instruction  from  2*f*b*w  to  f*b*i#,  a 
saving  of  f*b*w  bits. 

Let  the  relative  frequency  of  branch  instructions  be  x 

(in  one  sample  x  =  0.1  (Alexander  72)).  Changing  the 

addressable  unit  from  2*b  bits  to  b  bits  adds  an  average  of 

1  bit  per  branch  address,  or  x  bits  per  instruction. 

Therefore,  in  the  instruction  stream,  an  addressable  unit  of 

b  bits  is  preferable  to  an  addressable  unit  of  2*b  bits  if 

« 

f*b*w  >  x,  i.e.,  if  b  >  x/(f*w).  Typically,  the  right  side 
of  this  inequality  has  a  value  in  the  range  0.05  to  0.5, 
indicating  that,  for  instruction  lengths,  bit- var iabilit p  is 
preferable  to  any  coarser  unit. 

The  above  conclusion  is  based  on  the  assumption  that 
the  redundancy  in  each  field  of  an  instruction  is 
proportional  to  the  upper  bound  on  the  redundancy  in  a 
minimum  redundancy  encoding  of  the  field.  But  since  the 
class  of  branch  addresses  is  infinite,  there  is  no  algorithm 
for  finding  a  minimum  redundancy  encoding,  and  we  are  left 
to  our  coding  ingenuity.  When  that  is  inconsistent  for 
changing  b,  w  will  not  be  constant.  When  our  ingenuity  is 
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weak,  w  will  be  large  (>1) ,  strengthening  the  conclusion 
that  b  should  be  as  small  as  1  bit. 

The  regularity  in  branch  addresses  that  will  alio#  us 
to  take  advantage  of  more  than  one  address  length  has  been 
observed  by  several  people  (Alexander  72,  Saal  5  Shustek 
72).  It  is  simply  that  short  jumps  are  more  common  than 
long  ones.  From  his  sample  of  programs  for  the  IC7000,  Saal 
reports:  "The  results  of  our  analysis  have  indicated  that 
approximately  50  percent  of  all  branches  change  the  program 
counter  by  less  than  eight  locations.”  (Saal  S  Shustek  72) 
If  branch  addresses  are  expressed  relative  to  the 
instruction  pointer  (e . g .  ,  PDP-11),  instruction  sequences 
will  be  location  independent  (relocatable)  without  any  new 
registers  (the  instruction  pointer  is  there  anyway)  which 
have  to  be  specially  reset  on  program  relocation.  (A  seconl 
form  of  address  becomes  necessary  to  transfer  to  separately 
placed  instruction  sequences.  A  call  instruction  with  a 
data  address,  and  a  return  instruction,  may  be  used  for  this 
purpose.)  An  appropriate  representation  of  relative  branch 
distances  can  be  chosen  to  match  the  distribution  in  the 
manner  discussed  in  Section  4.1.  Noticing  that  the 
distributions  are  forward  biased,  we  choose  to  express  the 
branch  distances  relative  to  the  updated  iastruction 
pointer,  making  branch  distance  zero  a  no-operation. 
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A  major  problem  confronts  the  user  of  a  machine  <fith 
more  than  one  length  of  branch  address:  During  instruction 
assembly,  how  much  space  should  be  reserved  for  a  forward 
branch?  Richards  has  given  a  solution  for  two  sizes  that  is 
non-linear  in  the  number  of  branches  (Richards  71),  ani  for 
more  sizes  the  problem  appears  to  require  a  combinatoric 
solution-  The  cause  of  this  complexity,  however,  is  simply 
the  notorious  qo  to.  The  human  reader's  difficult/  in 
understanding  an  unstructured  program  parallels  the 
assembler* s  difficulty  in  computing  branch  addresses.  If  we 
restrict  ourselves,  as  Dijkstra  and  others  suggest  (Dijkstra 
68) ,  to  disciplined  control  structures  such  as 
if. -  -  then . . else. - ,  while . .do. . ,  and  repeat. .until. . ,  the 
solution  becomes  linear  in  the  number  of  branches.  If  cole 
is  generated  "from  inside  to  outside"  (Rudmik  74),  that  is, 
in  the  order  that  reductions  are  performed  iaring  a 
canonical  parse,  then  the  restriction  to  structired 
programming  ensures  that  branch  addresses  at  each  stage  can 
be  calculated  knowing  only  the  size  of  inner  structures. 

The  more  general  loop  construct,  an  unbounded  iteration 
with  exit  statements,  poses  no  new  problems  (the  branch 
address  for  the  last  exit  must  be  calculated  first) ;  but  the 
more  general  case  selection  construct  does.  Its  form  is: 

case  n  of  SI ,S2 , . . . , Sm 

where  n  is  an  expression  and  S1,..,Sm  are  statements  or  cole 
segments,  and  it  is  somehow  guaranteed  that  1  <  n  <  m.  if  it  h 
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our  relative  branch  addressing,  the  most  obvious  cods  for 
this  construct  is: 

calculate  n 

. - branch  n  instructions 


|  branch - 1 

!  -  I 

1 - »  -  I 

I 

branch - 1  | 

S 1  < — - H 

, - branch  | 

I  •  I 

I  •  I 

I  -  I 


1 - branch  (may  be  removed  by  optimization) 

i - > 

There  are  two  problems  with  the  above.  First,  if  we 
know  that  we  are  in  a  branch  table,  only  the  branch 
addresses,  not  the  branch  operation  codes,  are  neeled. 
Second,  if  the  code  is  to  remain  pure  (reentrant, 
unchanging) ,  we  have  no  way  to  represent  the  changing  branch 
into  the  branch  table.  One  solution  is  a  new  "branch- 
indirect”  instruction  with  an  indexed  data  address  operand, 
specifying  a  table  of  branch  addresses, 
calculate  n 

branch-indirect  Brtab(n) 

S1< - K  -r-  _  | 

, - branch  |  .  | 

I  -  I  •  I 

I  -  I  •  I 

I  -  I _ ._l 

,  sm< - — . 

| - branch  (may  be  removed) 

' - > 

This  is  very  similar  to  the  usual  IBM  360  solution 
(e.g. ,  XPL) ,  but  here  the  addresses  in  the  branch  table  are 
relative  to  the  instruction  pointer,  therefore  short. 
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The  branch-indirect  instruction  can  be  used  for  a  go  12 
statement,  whether  its  object  is  a  label  constant,  or  libel 
variable.  In  fact,  the  branch  table  for  a  case  statement  is 
simply  a  structured  constant.  The  branch  instructions 
located  between  the  alternatives  of  a  case  statement  are 
simply  a  dispersed  and  inverted  form  of  the  branch  taule, 
and  as  such  they  are  completely  redundant.  This  redunlancy 
can  be  exploited  by  a  "case"  instruction  with  two  operands; 
one  operand  is  the  number  of  alternatives,  and  the  other  is 
an  implicitly  indexed  data  address  specifying  a  branch  table 
as  above. 

calculate  n 

case  m,Brtab  _ 

S1< 

S2< 


Sm< 

< 

This  branch  table  has  one  extra  entry,  pointing  to  the  end 
of  the  case  statement.  When  the  case  instruction  is 
executed,  the  location  of  the  end  of  the  case  statement  is 
computed  and  pushed  onto  a  stack;  the  location  of  the 
alternative  following  the  one  to  be  executed  is  also 
computed  and  pushed  onto  the  same  stack;  and  control  is 
transferred  to  the  alternative  to  be  executed.  The  enl  of 
the  alternative  being  executed  is  recognized  by  continually 
comparing  the  instruction  pointer  to  the  stack  top;  control 
is  then  transferred  to  the  location  specified  by  the  naxt- 


-51- 


to-top  stack  entry,  and  the  stack  is  popped  twice.  A 
simpler  mechanism,  which  partially  exploits  the  redundancy 
and  avoids  comparing  the  instruction  pointer  to  the  stack 
top,  is  to  place  an  "end-of-alternat ive"  instruction, 
without  operands,  after  each  alternative  in  the  case 
statement- 

All  the  same  considerations  that  apply  to  "op 
constant",  "op  variable",  "op  constant (n) ",  "op  variable (n) " 
for  any  operator  apply  to  branching,  except  one:  the 
statement  go  to  <label  constant>  must  be  treated  as  go  to 
<label  variable>  in  order  to  avoid  the  combinatoric 
complexity  of  including  unstructured  branch  addresses  in  the 
code.  Hence  a  branch  address  may  be  considered,  not  as  a 
kind  of  data  address,  but  as  a  kind  of  constant  data. 

4.3  Data  Addresses 

A  "data  address"  may  simply  identify  a  variable  (e.g., 
"variable  number  3"),  or  it  may,  in  addition,  give  some 
location  information,  or  locate  a  variable  exactly.  If  all 
variables  are  allocated  the  same  amount  of  space,  the  above 
two  extremes  are  identical  (e.g.,  B5500) .  To  identify 
variables,  "data  addresses"  need  only  distinguish  variables 
that  are  simultaneously  accessible.  If  variables  are  not 
all  allocated  the  same  amount  of  space,  and  instructions  do 
no  more  than  identify  variables,  their  locations  must  be 
found  by  an  associative  search,  using  a  variable’s  identity 
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as  key.  If  instructions  give  some,  but  not  complete, 
location  information,  this  information  may  be  used  as  a 
’’hash  index”  to  make  the  search  less  expensive.  In  any 
case,  evaluation  of  this  issue  depends  on  technology  and  on 
available  resources  (static  versus  dynamic  costs) ,  whica  we 
are  not  prepared  to  undertake  in  this  thesis.  We  assume  in 
the  sequel  that  data  addresses  locate  variables  exactly.  In 
this  section  we  address  ourselves  to  three  questions:  How 
many  data  addresses  should  an  instruction  have?  What  form 
should  they  take?  What  is  the  appropriate  addressable  unit 
for  data? 

The  number  of  data  addresses  per  instruction  is  oae  of 
the  most  distinctive  characteristics  by  which  we  classify 
computer  architectures.  In  a  zero  address  machine  (B6730) , 
the  operands  are  assumed  to  be  on  the  top  of  a  stack,  and 
the  result  of  an  operation  is  placed  on  the  top  of  the 
stack.  Even  on  a  zero  address  machine  there  are  usually  one 
or  more  instructions  that  include  an  explicit  data  address, 
for  loading  a  stored  value  onto  the  stack,  and  for  storing 
the  result  of  an  operation.  On  single- accumulator  machines, 
one  data  address  is  normal.  So-called  "one-plus-one” 
address  machines  have  several  registers  that  may  be  used  as 
operands  or  locations  for  results  of  operations;  one  address 
specifies  a  register,  the  other  specifies  a  location  in  main 
memory.  Machines  with  two  or  three  data  addresses  are  not 
uncommon,  but  we  know  of  none  with  more  than  three.  rtiese 
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formats  are  not  mutually  exclusive,  and  many  machines  have 
more  than  one  instruction  format. 

The  need  to  decide  how  many  data  addresses  instructions 
will  have  arises  in  machines  with  a  fixed  instruction 
length.  But  with  variable- length  instructions,  no  general 
decision  is  needed;  we  may  have  one  operation  with  few 
operands,  and  another  with  many.  The  appropriate  complesity 
of  operations  is  discussed  in  Section  4.4. 

Registers  are  used  as  data  locations,  for  operands  and 
results,  in  two  ways: 

(a)  as  temporary  storage  for  unnamed  intermediate 
results 

(b)  as  a  temporary  or  permanent  residence  for 
frequently-used  variables 

Compilers  commonly  use  registers  as  temporary  storage 
in  a  totally  predictable  manner,  as  in  the  following 
example.  A  and  B  stand  for  data  addresses  in  main  memory,  x 
and  y  are  registers.  The  semantics  of  some  typical 
register-storage  operations  are: 


Load : 

L 

x,  A 

X 

<-  A 

Store: 

ST 

x,  A 

A 

<-  X 

Binary  Op: 

BOP 

x,  A 

X 

<  x  BOP  A 

Unary  Op: 

UOP 

x,  A 

X 

<-  UOP  A 

In  this  example,  the  relationship  between  the  registers  in 
the  instruction  pair 
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OP  1  x,A 
0P2  y,B 

is  as  follows: 

0P2 

_ 1_L _ 1_ST _ J _ BOP _ J _ UOP _ 

OP1  _L _ I  y-x+ 1  \  y=x _ J _ y=x _ L_yf x±l_ 

ST _ |  y=x _ |  y=x-1  1  y-x- 1  |  y-x _ 

BOP  I  y=x+ 1  1  y=x _ J _ y=x _ |  y=x+1 

OOP  1  y=x+1  1  y-x _ 1  y=x _ L_Zz£±l_ 

For  register-register  operations  we  have: 

operation  1 _ semantics  I  relation 

_BOP_xxy _ 1  x  <-  x  BOP  y  I  y-x-*- 1 _ 

POP  x,y _ _ 1  y=x _ 

where  x  depends  on  the  previous  instruction  as  above,  and 
the  following  instruction  depends  on  y  as  above. 

This  relationship  between  registers  results  from 
treating  them  as  an  evaluation  stack  (Haley  62,  RandelL  & 
Russell  64) .  Departures  from  stack  techniques  are  usually 
the  result  of  inconsistent  register  capabilities  (e.g.,  IBM 
360  Divide  requires  the  dividend  to  be  an  even-odd  register 
pair) .  The  main  justification  of  stack  architectures  has 
been  a  saving  in  space  (Haley  62,  Creech  70)  over  siagle 
accumulator  machines,  which  must  store  and  retrieve 
temporary  results,  and  over  register  machines,  which  Bust 
state  registers  explicitly  even  when  the  stack  regularity 
makes  their  use  as  temporary  storage  totally  predicta  ale. 
By  keeping  the  top  portion  of  the  stack  (the  active  part)  in 
registers  and  the  remainder  in  main  memory,  stack  machines 
appear  to  have  a  large  amount  of  temporary  storage  at 
(almost)  main  memory  prices  but  (almost)  register  speeds. 
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The  second  use  of  registers  as  data  locations  is  for 
temporary  or  permanent  residence  of  frequently- ased 
variables.  As  such  they  are  the  top  grade  in  a  hierarchy  of 
memories  that  may  include  semi-conductors,  core,  drums  or 
disks,  tapes,  and  people.  Normally  the  addressing  spices 
for  each  grade  of  memory  are  separate,  and  movement  between 
different  grades  is  accomplished  by  different  instructions: 
load  and  store  instructions  move  data  between  registers  and 
main  memory,  while  input/output  instructions  move  lata 
between  main  memory  and  lower-grade  memories.  This  thasis 
contends  that  there  is  a  cost  involved  in  fixing,  an  the 
machine-language  level,  the  number  of  grades  of  memory,  or 
their  sizes.  Although  this  cost  is  difficult  to  estimite, 
it  becomes  painfully  clear  when  a  change  is  needed.  This 
usually  means  a  complete  redesign  of  the  machine,  and 
obsolescence  of  all  programs,  or  at  least  of  all  compilars, 
for  the  old  machine.  If  registers  are  used  to  hold  the 
variables  that  are  referenced  most  frequently,  they  snould 
have  shorter  addresses  than  main  memory  locations,  but  nor  a 
separate  addressing  space.  If  secondary  memory,  main 
memory,  and  registers  used  to  hold  variables  are  all  part  of 
a  continuous  addressing  space,  with  a  uniform  notation  for 
data  movement,  the  programs  can  take  instant  advantage  of:  an 
increase  in  high-grade  memory;  conversely,  all  programs  for 
a  machine  of  this  design  can  run,  although  more  slowly,  on 
an  inexpensive  configuration. 
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An  addressing  space  that  is  larger  than  main  memory  is 
not  a  new  idea;  virtual  memory  was  introduced  by  Manchester 
University  on  their  ATLAS  computer  (Kilburn  et  al.  62)  .  But 
to  our  knowledge,  a  uniform  treatment  of  all  grades  of 
memory  has  yet  to  be  carried  out. 

It  may  be  advantageous,  for  the  purpose  of  constructing 
address  registers,  but  not  at  the  machine-language  level,  to 
acknowledge  that  memory  is  finite.  Although  it  is  a  snail 
computer,  the  Bl700*s  addressing  space  is  2*4  bits  (Wilner 
72a&b) .  This  virtual  memory,  although  finite,  is  in  no 
danger  of  becoming  inadeguate  in  the  foreseeable  future. 

An  established  technique  for  shortening  data  addresses 
and  at  the  same  time  making  data  segments  relocatable  *ith 
very  little  trouble  is  to  express  each  address  as  an  offset 
added  to  the  contents  of  a  register.  It  may  be  called 
"base,  displacement”,  with  the  programmer  being  responsible 
for  explicitly  maintaining  the  registers  (e.g.,  IBM 
System/360) ,  or  it  may  be  called  "lexic-level,  ocder- 
n umber”,  with  the  hardware  maintaining  the  registers  on 
block  or  procedure  entry  and  exit  (e.g..  Burroughs  B6730). 
The  latter  terminology  is  motivated  by  a  macnine 
architecture  for  block-structured  languages  that  organizes 
data  items  into  activation  records  (Randell  &  Russell  54)  . 
At  any  time  during  execution,  one  activation  record  will  be 
addressable  for  each  lexic-level  in  the  source  program  up  to 
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and  including  the  current  level,  so  one  register  for  each 
lexic-level  is  maintained  with  the  address  of  the  base  of 
the  appropriate  activation  record.  (These  bases  are  not 
necessarily  all  held  in  registers.  At  any  one  time,  the 
Burroughs  B5500  holds  only  two  in  registers,  and  the 
remainder  in  core.  The  B6700  holds  all  bases  in  registers, 
to  its  maximum  of  32,  and  the  IBM  360  leaves  this  decision 
to  the  compiler  writers.)  The  habits  of  users  of  such 
systems  have  been  studied  (Wortman  72)  ,  and  this  knowledge 
can  be  used  to  advantage  in  the  encoding  of  these  addresses. 
The  B6700,  for  example,  uses  a  variable-length  field  for  the 
lexic-level  and  for  the  order-number  such  that  their  total 
length  is  fixed.  Further  improvement  is  possible  by 
allowing  each  field  to  vary  separately. 

To  keep  instructions  pure  (reentrant,  unchanging)  when 
dealing  with  structured  data,  a  machine  designer  must 
provide  the  ability  to  index  a  data  address,  that  is,  to 
specify  a  computed  offset  to  be  added  to  the  address.  On 
many  machines,  some  data  addresses  include  an  "index 
register"  field  (IBM  S/360) .  But  the  registers  provided  are 
not  equally  used  for  this  purpose,  and  some  studies  have 
shown,  for  particular  cases,  that  almost  half  the  time  no 
index  is  used  at  all  (Alexander  72)  .  Using  the  stack 
evaluation  technique,  the  register  holding  a  computed  iniex, 
as  with  any  other  computed  quantity,  will  be  the  current  top 
of  the  stack,  and  it  need  not  be  specified  explicitly.  For 
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a  simple  index  variable,  explicit  specification  can  be 
advantageous  (see  Section  4.4.2). 

The  appropriate  addressable  unit  for  data  may  be  found 
in  a  manner  similar  to  that  for  instructions.  As  was  the 
case  there,  a  coarse  addressable  unit  saves  spare  in  the 
address  field,  and  costs  space  in  the  addressed  item. 

Let  the  distribution  of  lengths  allotted  to  variiDles 
be  f.  Let  the  addressable  unit  of  data  be  b  bits.  Then  the 
average  number  of  addressable  units  per  variable  is 
inf 

a  =  SUM[ s/b  ]*f (s) 
s=1 

where  [  ]  means  rounded-up  or  ceiling,  and  "inf"  denotes  an 
upper  limit  of  infinity.  Using  the  representation  of 
Appendix  B. 2  for  addresses,  and  assuming  that  the 
probability  of  referencing  a  variable  is  independent  of  the 
variable,  the  average  space  per  address  is  approximately 

n 

d  =  (1/n) *SUM  2*log ( i*a) 
i=  1 

=  2*log  (a)  +  (2/n)  *log  (n ! ) 

=  approx.  2*log  (n*a)  for  large  n 
where  n  is  the  total  number  of  variables,  and  the  logarithm 
base  is  2. 

Suppose  there  are,  on  average,  x  data  addresses  per 
data  item.  For  branch  addresses  we  had  x  <<  1  since  nost 

instructions  are  not  the  object  of  any  branch.  But  all 


-59- 


useful  variables  are  referred  to  at  least  once,  so  for  data 
addresses  x  >1.  In  one  sample,  x  =  10.4  (from  production 
frequencies,  Alexander  72) .  The  total  space  for  addresses 
and  variables,  per  variable,  is 
S  =  x*d  +  a*b 
with  units 

(space/vbl)  =  (addresses/vbl) * (space /address)  + 

(addressable  units/vbl) * (space/addressable  unit) 

Given  f  and  x,  we  should  choose  b  to  minimize  S.  It  can  be 

seen  that  the  minimum  will  be  independent  of  n.  Let  us  Look 

at  two  extreme  examples. 

For  f  (s)  =1  if  s=c  (a  constant) 

0  otherwise 

S  is  minimum  when  b=c,  independent  of  x. 

For  f  (s)  =  2**-s  and  x  =  10.4, 

S  is  minimum  when  b  =  5  (see  Appendix  C) . 

For  any  distribution  of  lengths  allotted  to  variables,  and 
average  number  of  addresses  per  variable,  we  can  find  the 
best  addressable  unit  in  a  similar  manner.  The  appropriate 
space  to  allot  a  variable  is  discussed  in  Section  5. 

If  instructions  imply  the  types  of  operands,  this 
information  can  be  used  to  advantage  in  the  specification  of 
data  addresses.  By  maintaining  a  separate  base  for  each 
type,  we  may  use  the  same  specification  for  a  variable  of 
one  type  that  is  used  for  a  variable  of  another,  witaout 
confusion.  For  example,  at  machine-langauge  level  we  may 


-60- 


have  a  logical  x  and  an  integer  x  in  the  sane  scope. 
Furthermore,  we  may  choose  the  appropriate  addressable  unit 
for  a  type  according  to  the  distribution  of  lengths  of 
variables  of  that  type,  independent  of  the  addressable  unit 
for  other  types.  A  different  exploitation  of  the  type 
dependency  within  an  instruction  is  discussed  in  Section 
4.4. 2. 


4 . 4  The  Rest 

The  final  part  of  instructions  consists  of  a  smalL  set 
of  operations,  and  a  small  set  of  operand  types  anl  formats. 
For  convenience  we  shall  refer  to  this  portion  of  an 
instruction  as  its  op-code.  Since  this  class  is  finite,  we 
may  use  algorithmic  means  of  matching  the  code  to  the 
entropy  that  were  not  applicable  to  the  other  classes.  For 
op-codes  that  appear  with  independent  probabilities,  Huffman 
has  given  an  algorithm  for  computing  a  set  of  codes  whose 
redundancy  is  minimum  over  all  possible  codes  (Huffman  52) . 
It  has  been  observed,  however,  that  op-codes  are  not 
independent  (Wortman  72,  Alexander  72,  Foster  et  al.  71), 
and  efforts  have  been  made  to  take  advantage  of  the 
dependencies. 

Wortman  was  able  to  improve  the  op-code  set  of  the 
Student  PL  machine  (Wortman  72)  by  inventing  new  op-codes  to 
replace  certain  pairs  of  old  ones  wherever  they  occur  in  a 
program.  The  pairs  were  chosen  on  the  basis  of  high 
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frequency,  and  compatible  semantics,  i.e.,  pairs  that  fit 
"naturally"  together;  the  result  of  the  combinations  was  a 
reduction  in  the  space  required  for  an  average  program. 
This  technique  can  be  made  algorithmic  by  raplacing, 
wherever  it  occurs  ir.  the  sample,  that  pair  that  reduces  the 
information  content  the  most  (Appendix  D) ,  then  repeating 
until  some  prespecified  limit  is  reached.  We  shall  call 
this  method  "iterative  pairing".  As  a  heuristic  for 
recognizing  which  pair  reduces  information  content  the  most, 
we  may  choose  the  most  common  pair.  This  heuristic  is  well- 
suited  to  machines  with  fixed-length  fields,  since  it  tends 
to  produce  op-code  combinations  that  have  nearly  equal 
frequencies.  Unfortunately,  there  may  be  no  pair  whose 
replacement  reduces  the  information  content;  it  may  actually 
increase  for  any  or  all  pairs.  Furthermore,  gaining  the  most 
benefit  from  a  limited  number  of  replacements  may  require 
choosing,  at  intermediate  stages,  pairs  other  than  those 
that  reduce  information  content  the  most  or  those  that  are 
most  frequent. 


If  we  replace  all  pairs  of  op-codes  by  new  op-codes,  or 
in  general,  all  m-tuples  by  compound  op-codes,  then  the 
space  required  for  a  minimum  redundancy  encoding  of  the  m- 
tuples  will  not  increase,  and  will  decrease  if  there  are 
dependencies  of  order  m  or  less  (Appendix  A. 1).  This  coding 
method  corresponds  to  the  estimates  of  entropy  based  on  m- 
tuple  frequencies 


(G  function 


of 


Section  3.3). 
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Implementation  considerations  limit  the  size  of  the  op-code 
set,  which  may  grow  exponentially  with  m.  In  practice, 
however,  the  growth  is  much  slower:  in  one  study,  of  the 
1021  possible  10-tuples  of  op-codes  on  the  CDC3600,  fewer 
than  7000  had  non-zero  frequencies,  indicating  that  there 
are  important  dependencies  of  order  10  or  less  (Foster  72) . 
The  advantage  of  iterative  pairing,  in  spite  of  its 
uncertainty,  is  that  we  may  take  account  of  important  nigh- 
order  dependencies  before  less  important  low-ocder 
dependencies,  and  therefore  gain  more  benefit  from  fewer  new 
instructions. 

Foster  and  Gonter  have  suggested  a  method  of  shortening 
the  op-code  field  that  takes  advantage  of  inter-instruction 
dependencies  (Foster  &  Gonter  71).  In  their  method,  each 
operation  is  allowed  k  successors  plus  an  escape;  that  is, 
following  any  operation,  only  its  k  most  frequent  successor 
operations  are  given  a  code,  plus  one  op-code  for  all  n-k 
other  operations.  The  "operand”  of  the  escape  code  is  used 
to  set  the  state  of  the  machine  as  it  would  be  if  an 
operation  which  has  the  desired  successor  had  just  neen 
performed.  These  codes  are  called  "conditional",  since  an 
op-code  can  be  decoded  only  by  knowing  the  preceding 
operation.  Since  Foster  and  Gonter  used  a  fixed-length  op¬ 
code  field,  their  method  can  save  space  only  if  k  <  n  -  1. 
With  a  variable-length  field  there  is  no  advantage  in 
keeping  k  <  n,  so  we  shall  take  k  =  n.  We  make  use  of  the 
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variable-length  fields  by  giving  the  op-code  set  a  minimum 
redundancy  encoding  for  each  possible  predecessor  op-code. 
In  keeping  with  our  "no  overflow"  policy,  we  advocate 
inclusion  of  a  zero-frequency  non-operation  in  the  op-code 
set.  This  produces  an  open-ended  code  for  future  expansion, 
at  a  cost  of  one  bit  on  the  least  frequent  operation. 

An  immediate  generalization  of  their  method  makes  each 
op-code  conditional  upon  the  preceding  m- 1  operations; 
knowing  the  m-tuple  frequencies  we  can  generate  the 
appropriate  codes.  This  corresponds,  in  real  coding  terms, 
to  the  mth  estimate  of  class  entropy  based  on  conditional 
symbol  entropy  (F  function  of  Section  3.3).  Like  m-tupling, 
this  method  carries  the  guarantee  which  iterative  pairing 
lacks:  it  will  not  increase  the  required  space,  and  *  ill 
reduce  it  if  there  are  dependencies  of  order  m  or  less 
(Appendix  A. 2).  For  each  m  (greater  than  1)  this  method  is 
more  effective  in  reducing  space  than  compounding 
instructions  into  m-tuples  (Appendix  A.. 3).  Furthermore, 
conditional  coding  is  more  readily  viewed  as  just  a  coding 
technique,  not  as  an  increase  in  the  op-code  set. 

Eastwood  has  proposed  "dynamically  variable"  encoding 
for  the  representation  of  machine-language  instructions 
(Eastwood  73) .  By  "dynamically  variable"  he  does  not  mean 
that  code  changes  during  execution,  but  that  an  instruction 


may  change  its 
code. 


representation  from  place  to  place  in  the 


"By  creating  a  representation  optimally  for  just  those 
operations  and  operand  references  used  in  each  critical 
program  sequence,  such  as  an  inner  loop,  a  significant 
local  compression  of  the  instruction  information  caa  be 
obtained.  The  primary  goal  is  not  the  reductioa  of 
storage  space  but  the  increase  in  effective  bandwidth 
for  the  transmission  of  information  from  storage.  This 
should  result  in  an  overall  speedup  of  execution." 

The  techniques  suggested  by  Eastwood  for  encoding 

instructions  are: 

1.  Condensation  of  op-codes.  He  does  not  suggest  any 
method  for  achieving  this. 

2.  Variable-length  encoding  of  address/offset 
information.  By  this  he  means  the  suppression  of 
leading  zeros.  He  will  need  some  method  for  encoling 
the  length  of  the  field,  as  discussed  in  Section  4.1. 
All  fields  of  an  instruction  can  benefit  from  variaoie- 
length  encoding. 

3.  Pronominal  addressing.  This  means  giving  specially 
recognized,  short  addresses  to  recently  referenced 
variables.  In  Section  4.3  we  suggested  the  related 
idea  that  high-grade  memory  (registers)  used  as 
temporary  residence  of  frequently-used  variables  shculd 
have  shorter  addresses  than  main  memory  locations,  but 
not  a  separate  addressing  space. 

4.  Field  citation.  This  means  an  indication  that  a 
field  is  identical  to  a  field  in  a  previous 
instruction,  i.e.,  pronominal  fields  in  general. 
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5.  Dynamic  register  assignment.  When  taken  to  an 
extreme,  this  will  result  in  a  stack  machine. 

6.  Instruction  fetch  macro  seguences. 

Eastwood  refers  to  Foster  and  Gonter's  conditional  op-coles, 
but,  claiming  that  his  approach  "takes  no  advantage  of  the 
type  of  redundancy  which  Foster  and  Gonter  exploit",  he 
concludes  that  "these  techniques  are  independent".  East*ood 
has  not  proposed  algorithmic  means  for  determining  the  local 
character  of  an  instruction  sequence  so  that  a  compiler  can 
produce  locally  optimal  representations  of  op-codes,  but 
presumably  this  local  character  is  determined  by  what  the 
preceding  op-codes  are.  Eastwood's  technique,  as  it  applies 
to  op-codes,  is  not  independent  of  but  equivalent  to 
conditional  coding. 

4.4.1  Conditional  Encoder /Decoder 

Figure  3a  shows  the  inputs  and  outputs  that  a 
conditional  encoder  must  have,  and  figure  3b  shows  the 
inputs  and  outputs  of  a  conditional  decoder.  The  encoder  is 
a  part  of  the  instruction  assembler,  which  may  be  entirely 
software,  or  may  be  firmware  or  hardware  assisted.  After  an 
operation  is  encoded,  producing  the  code  and  length  of  code 
as  output,  the  operation  register  is  shifted  into  the 
context  register,  and  then  refilled  with  the  next  operation 
to  be  encoded.  The  representation  of  an  operation  in  the 
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operation  and  context  registers  is  fixed-length  and 
unconditional,  and  is  entirely  internal  to  the  assembler. 

For  sufficient  execution  speed,  the  decoder  must  be 
firmware  or  hardware.  It  produces  a  fixed-length, 
unconditional  representation  of  the  operation  as  output, 
along  with  the  length  of  the  input  op-code.  The  output 
representation  is  further  decoded  in  the  usual  manner;  this 
representation  is  entirely  internal  to  the  decoder,  although 
it  may  resemble  that  used  in  the  encoder.  After  aa  op-oode 
is  decoded,  the  operation  register  is  shifted  into  the 
context  register,  and  a  new  op-code  is  shifted  into  the  oode 
register. 
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where  [ ]  means  rounded  up  (ceiling) 
z  =  length  of  the  longest  code 
n  =  number  of  operations 
m  =  order  of  conditional  code 


Figure  3:  Conditional  Encoder  and  Decoder 
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The  encoding  and  decoding  procedures  described  above 

are  adequate  for  sequential  flow  of  control,  but  reguire 

embellishment  to  handle  branching.  We  shall  assume  for  the 

moment  that  every  instruction  in  an  instruction  sequence  may 

be  reached  during  execution  by  at  least  one  path  (an 

intelligent  compiler  can  guarantee  this) .  If  an  instruction 

has  exactly  one  possible  predecessor  instruction,  the 

context  needed  for  encoding  or  decoding  it  is  obtained  from 

the  context  used  by  its  predecessor  according  to  the 

procedures  described  above.  But  when  an  instruction  has 

more  than  one  predecessor,  conditional  coding  requires  that 

all  its  predecessors  supply  it  with  identical  contexts. 

Consider  first  the  if. .then. . else. .  control  construct. 

if  el  then  S 1 
else  S2 
S3 

Diverging  control  causes  no  problems;  the  instruction 


sequences  SI 

and  S2 

each  have 

a  single 

predecessor ; 

the 

conditional 

branch 

following 

evaluation 

of 

the  expres 

sion 

el.  But  merging  control  at  S3 

requires 

that  SI 

ani 

S2 

supply  the 

same 

context 

to  S3. 

SI 

ends 

with 

an 

unconditional 

branch 

.  S2  can 

be  made 

to 

end 

with 

an 

unconditional  branch  of  distance  zero.  Before  we  draw  any 
conclusions  let  us  examine  the  while- -<|o..  control 
construct. 

S4 

while  e2  do  S5 
S6 
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The  head  of  the  loop,  expression  e2,  is  preceded  dicing 
execution  by  both  the  previous  statement  group  S4,  and  the 
body  of  the  loop  S5.  S5  ends  with  an  unconditional  branch, 
and  S 4  can  be  made  to  end  with  an  unconditional  branch  of 
distance  zero-  The  body  of  the  loop  S5  and  the  instruction 
sequence  S6  which  follows  the  loop  are  each  preceded  during 
execution  only  by  evaluation  of  the  expression  e2r  which 
ends  with  a  conditional  branch. 

From  the  above  two  constructs,  the  following  riles 
emerge : 

During  program  assembly,  the  else  part  of  an  if  then  else 
and  the  instructions  prior  to  the  head  of  a  while  loop 
must  be  made  to  end  with  a  branch  of  distance  zero. 
The  instructions  following  an  if  then  else,  and  the 
instructions  in  the  head  of  a  while  loop  must  not  be 
encoded  in  the  context  of  the  textually  preceding 
instructions;  instead  a  standard  context  must  be 
substituted  at  that  point.  The  instructions  in  the 
else  part,  and  the  instructions  following  a  while  loop, 
may  use  the  context  of  the  expression  in  the  if  pact  or 
loop  header. 

During  program  execution,  the  unconditional  branch  Bust 
place  this  standard  context  in  the  decoder's  context 
register.  A  conditional  branch  does  not  modify  the 


context  register. 
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The  standard  context  may  be  chosen  arbitrarily,  but 
once  chosen,  it  should  be  used  consistently  for  all  merging 
control  paths.  Encoding  for  the  first  instruction  Ln  a 
procedure  should  assume  a  standard  context,  and  the  call 
instruction  should  establish  this  context  during  execution. 
The  instruction  after  a  call  could  assume  the  standard 
context,  which  the  return  instruction  could  establish,  or 
context  may  be  maintained  across  a  procedure  call  by 
stacking  and  re-establishing  context  on  procedure  call  and 
return.  In  general,  any  arbitrary  control  construct  can  be 
handled  by  using  a  branch  of  distance  zero  to  establish  a 
standard  context  in  sequential  flow  whenever  it  is  needed. 

The  cost  of  the  encoder/decoder  places  a  practical 
limit  on  the  order  of  conditional  coding,  that  is,  on  the 
amount  of  context  used  to  encode  and  decode  instructions. 
Machine  designers  must  decide  how  much  buyers  are  willing  to 
pay  for  performance  gains.  Also,  subject  to  fan-in 
limitations  of  circuitry,  the  speed  of  a  wide  decoder  could 
be  slow,  and  ultimately  affect  the  speed  of  the  machine.  We 
point  out  also  that  the  increase  in  performance  diminishes 
as  the  order  of  conditional  coding  increases. 

The  frequency  of  merging  control  points  and 
unconditional  branches  may  affect  the  choice  of  decoder 
width.  If  the  frequency  of  merges  is  high,  the  space  saved 
by  increasing  the  order  of  conditional  coding  will  be  small. 
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If  the  frequency  of  branches  is  high,  the  increase  in 
effective  bandwidth  will  be  small.  According  to  statistics 
for  XPL  programs  on  the  IBM  360  (Alexander  72) ,  only  oqs  in 
12  instructions  is  an  unconditional  branch,  procedure  call, 
or  return,  hence  at  most  one  in  12  instructions  is  a  merge 
point.  This  frequency  is  low  enough  that  we  expect  its 
effect  on  performance  to  be  negligible. 

4.4.2  Op-code  Constitution 

To  perform  an  operation,  we  must  have  the  following 
information: 

1.  operation  specification;  e.g.,  add,  catenate, 
move,  compare 

2.  the  number  of  specified  operands;  e.g.,  0,  1,  2 

and  3  operand  adds  may  all  be  available  on  rhe 
same  machine 

3.  the  format  of  the  operand  specifications;  e.g., 

constant,  indexed  address,  indirect  address 

4.  the  type  of  operands 

5.  the  length  of  the  space  allotted  to  the  operanis 

6.  the  value  or  location  of  the  operands 

For  example,  the  IBM  S/360  op-code  "add  halfword”  informs  us 
that 

1.  the  operation  is  "add" 

2.  there  are  two  specified  operands,  of  whicti  the 

first  doubles  as  result  location 
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3.  the  first  operand  is  specified  in  register  focnat, 
and  the  second  in  indexed  base-displacement  format 

4.  the  operands  are  two's  complement  integers 

5.  the  first  operand  occupies  32  bits  and  the  second 
16  bits 

The  remainder  of  the  instruction  specifies  the  location  of 
the  operands.  Some  of  the  information  under  points  2  and  3 
must  be  specified  in  the  instruction  in  order  that  the 
information  under  point  6  can  be  interpreted.  But  when  an 
operand  is  specified  by  a  data  address  rather  than  by  a 
constant,  we  have  an  option  whether  to  put  the  information 
of  points  4  and  5  in  the  instruction  or  at  the  data  addrass. 
(This  assumes  that  we  outlaw  constructs  such  as  Fortran 
EQUIVALENCE  that  allow  data  to  be  interpreted  in  more  than 
one  way.)  The  following  paragraphs  suggest  why  we  might 
want  to  do  so,  and  how  to  do  so. 

A  variable  that  is  never  referred  to  by  the  instruction 
stream  is  a  mistake.  On  average,  each  variable  is  referred 
to  by  more  than  one  instruction  (statically  and 
dynamically)  .  This  suggests  that  all  information  aboit  a 
variable  that  does  not  change  from  instruction  to 
instruction,  such  as  its  type  and  the  length  of  its  allotted 
space,  ought  to  be  placed  with  the  variable  rather  than 
duplicated  in  all  instructions  that  refer  to  it. 
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Self-de 

scribing  data  has  been 

proposed  previaasly 

(Barton  70)  , 

but 

these  proposals  have 

added  redundancy  by 

duplicat ing 

type  information.  To 

remove  the  type 

information 

from 

an  instruction,  number 

the  operations  for 

each  data  type 

according 

to  frequency. 

For  example: 

type 

• 

• 

logical 

character 

number 

most 

1 

OR 

LOAD 

LOAD 

frequent 

2 

AND 

STORE 

STORE 

3 

NOT 

COMPARE 

ADD 

operation 

4 

LOAD 

CATENATE 

SUBTRACT 

number 

5 

STORE 

SUBSTR 

COMPARE 

6 

EQUIV 

MULTIPLY 

least 

7 

COMPLEMENT 

frequent 

8 

DIVIDE 

Operation  number 

3  is 

interpreted 

as  NOT  if  the  data  is 

logical,  as  COMPARE  if  the  data  is  character,  and  as  ADD  if 
the  data  is  numerical.  This  scheme  results  in  very  few  op¬ 
codes  indeed.  An  equivalent  view  of  this  scheme  is  that  it 
takes  advantage  of  a  dependency  between  the  op-code  and  iata 
address  fields  of  an  instruction.  When  a  data  address 
refers  to  a  number,  the  operation  is  bound  to  be  one  that 
operates  on  numbers. 

The  above  scheme  relies  on  the  type  dependency  between 
an  operand  and  op-code  in  an  instruction.  Conditional 
coding  relies  on  type  (and  other)  dependencies  betweea  the 
op-codes  of  neighbouring  instructions.  But  dependence  is  a 
transitive  relationship,  and  therefore  these  two  technijues 
are  fighting  over  a  common  dependency.  An  example  will  nake 


this  clear. 
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Suppose  there  are  three  op-codes,  T1,  T2  and  T3,  nose 
operand  is  of  type  T,  and  there  are  three  op-codes,  SI,  S2 
and  S3,  whose  operand  is  of  type  S.  Suppose  that  all  op¬ 
codes  appear  with  equal  frequency,  but  an  op-cods  is 
followed  by  another  of  the  same  type  four  times  as  often  as 
by  one  of  the  other  type. 

Scheme  1:  Op-code  includes  type. 


After  T 1 , 

T2,  or  T3, 

op- 

codes 

may  be  encoded  as 

operation 

probability 

code 

length 

T 1 

4/15 

00 

2 

T2 

4/15 

01 

2 

T3 

4/15 

10 

2 

SI 

1/15 

1100 

4 

S  2 

1/15 

1101 

4 

S3 

1/15 

1110 

4 

open 

0 

After  SI, 

S2 ,  or  S3, 

we 

have 

a  similar  table. 

The  average  op-code  length  is  12/5  bits. 

Scheme  2:  Type  information  resides  with  variable. 

After  any  operation,  all  operation  numbers  are  eqmlly 
likely. 


op  number 

probability 

code 

length 

1 

1/3 

00 

2 

2 

1/3 

01 

2 

3 

open 

1/3 

0 

10 

2 

The  average  op-code  length  is  2.  Each  variiDle 
requires  1  extra  bit  to  specify  its  type.  If  the 
number  of  references  per  variable  is  x,  then  the 
operation  and  type  information  require  2  ♦  1 /x  bits  per 
op-code. 
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Conclusion:  To  minimize  space, 

if  x  <  5/2  references  per  variable,  use  scheme  1; 

if  x  >  5/2  references  per  variable,  use  scheme  2. 

In  general,  the  decision  whether  to  include  type  information 
with  the  operations  or  operands  depends  on  the  distribution 
of  operations  and  the  number  of  references  per  operani  in 
the  intended  environment.  We  expect  that  much  of  the 
benefit  of  conditional  coding  may  be  achieved  more  simpl/  by 
the  latter  choice. 

Other  dependencies  between  op-code  and  data  address 
fields,  similar  to  the  type  dependency,  appear  in  procedure 
call  and  array  indexing  instructions.  For  example, 
excluding  procedure-valued  variables,  when  an  operand  is  a 
procedure,  the  operation  is  bound  to  be  call,  and  vice 
versa.  If  a  procedure  begins  by  identifying  itself  as  a 
procedure,  this  information  need  not  be  carried  by  every 
call.  (This  beginning  may,  if  advantageous  to  the 
addressing  structure,  be  a  descriptor  in  the  data  area.) 
For  a  procedure  that  does  not  return  a  value,  the  call  may 
consist  of  the  shortest  (i. e. ,  most  frequent)  op-code  that 
indicates  a  single  data  address  operand.  For  a  procelure 
that  returns  a  value  (function) ,  the  op-code  may  take  its 
interpretation  from  the  type  of  value  returned.  In  effect, 
we  are  allowing  instructions  such  as  "add  x"  to  refer  either 
to  a  number  x,  or  to  a  number-valued  function  x.  »hen  the 
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address  is  pursued  and  the  operand  found  to  be  a  procelure 
or  function,  further  interpretation  of  the  op-coda  is 
delayed  until  the  procedure  is  executed,  and  the  value  is 
computed.  On  return,  the  interrupted  instruction  may 
continue  with  the  computed  value  as  operand. 

On  a  machine  where  array  operations  (as  in  APL)  are  not 
single  machine  instructions,  indexing  may  be  implied  by  the 
data  address.  Most  present  machines  fall  into  this 
category.  An  instruction  such  as  "add  x"  may  be  usel  to 
mean  "add  indexed  x"  when  x  refers  to  a  self-describing 
array.  (As  with  procedures,  the  descriptor  may  be 
physically  separate  from  the  elements  if  this  is  found  to  be 
advantageous.)  The  location  of  the  index  is  implicitly  the 
current  top  of  the  evaluation  stack  (see  Section  4.3). 

We  now  consider  a  machine  with  array  operations,  where 
"add  x"  and  "add  indexed  x"  must  be  distinguished,  meaning 
"add-array  x"  and  "add-element-of  x"  respectively.  Should 
there  be  an  indexing  instruction,  or  should  indexing  bs  a 
subfield  in  other  instructions?  For  fixed-length 
instructions  the  answer  depends  on  how  often  one  wants  to 
index,  and  the  relative  sizes  of  an  index  field  and  iadex 
instruction.  But  for  variable-length  instructions,  the 
answer  is  independent  of  indexing  frequency. 

Suppose  there  are  n  operations  (not  including  indexing)  and 
the  probability  of  the  ith  operation  is  Pi.  Suppose  further 


-77- 


that  a  data  address  in  an  instruction  is  indexed  with 
probability  X,  independent  of  the  operation,  and  if  indaxed, 
the  probability  that  index  register  i  is  used  is  Qi. 

Scheme  1:  Include  an  index  field  in  each  instruction. 

Space  per  instruction  for  scheme  1: 
op-code  space  =  -  SUM  Pi  log  Pi 

index  field  space  =  -  (1-X)  log  (1-X)  -  SUM(X  Qi}log(X  Qi) 

total  space  =  -SUM  Pi  log  Pi  -  X  SUM  Qi  log  Qi 

-  X  log  X  -  (1-X)  log  (1-X) 

Scheme  2:  Include  an  indexing  instruction  to  modify  an 
address  in  the  following  instruction. 

Space  per  original  instruction  for  scheme  2: 
op-code  space 

=  ( 1 +X) [ -SUM  [ _Pi  log  _Pi  ]  -  _X_  log  _X  ] 

[  [i+x  i+x]  i+x  nx] 

space  for  operand  of  index  instruction 

=  -X  SUM  Qi  log  Qi 

total  space  =  -SUM  Pi  log  Pi  -  X  SUM  Qi  log  Qi 

-  X  log  X  +  (1  +  X)  log  (1+X) 
space  for  scheme  2  minus  space  for  scheme  1 

=  (1  +  X)  log  (1+X)  +  (1-X)  log  (1-X) 

>  0  with  equality  iff  X  =  0 
Conclusion:  Scheme  1  is  superior  to  scheme  2. 

The  above  conclusion  did  not  take  into  account  any  space 
gains  possible  by  conditional  coding,  nor  did  it  account  for 
the  fact  that  a  minimum  redundancy  encoding  may  require  more 
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space  than  the  information  content  of  the  fields  under 
consideration.  These  factors  will  be  considered  below* 

The  reason  that  scheme  2  requires  more  space  than 

scheme  1  is  that  scheme  2  allows  the  possibility  of  more 

than  one  consecutive  index  operation,  whereas  scheme  1  loes 

not.  On  the  B6700  a  pair  of  index  operations  is  illegal 

since  an  indexed  descriptor  cannot  be  indexed  again.  On  the 

Myriad  the  combination  may  be  used  to  access  something  of 

the  form  a  (b  (c) )  as  follows: 

index  c 
index  b 
access  a 

if  a  and  b  are  each  zero-origin  arrays  of  word  elements, 
i.e. ,  the  addressable  unit  on  that  machine.  The  same  effect 
can  be  achieved  on  a  machine  using  index  fields  as  follows: 
load  c 

load  indexed  b 
access  indexed  a 

Since  single  indexing  ability  is  sufficient  to  maintain  the 
purity  of  instructions  even  for  the  uncommon  case  of 
indexing  an  index,  it  is  doubtful  whether  the  extra  power  of 
scheme  2  is  worth  the  extra  space* 

Suppose,  contrary  to  experience,  that  operation  codes 
are  independent  of  preceding  operation  codes,  and  suppose, 
as  before,  that  the  probability  that  an  instruction  is 
indexed  is  independent  of  the  operation.  Under  these 
circumstances,  conditionally  coding  operations  makes  schemes 
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1  and  2  equivalent  (Appendix  E) .  Returning  to  reality  and 
acknowledging  that  operation  codes  are  not  independent,  but 
retaining  the  assumption  that  indexing  is  independent  of 
(other)  operations,  we  find  that  conditional  coding  of  ocder 
m  reduces  space  more  when  each  instruction  includes  an  iidex 
field  than  when  indexing  is  a  separate  operation. 
Essentially,  the  insertion  of  indexing  op-codes  lengthens 
the  sequence  of  instructions  so  that  fewer  dependencies  are 
accounted  for  by  conditional  coding  of  a  given  order.  ^s  m 
increases,  the  difference  between  the  two  schemes  decreases 
to  zero. 

Finally,  we  remove  all  independence  assumptions.  [Jader 
the  index  instruction  scheme,  conditional  coding  would 
automatically  take  advantage  of  any  dependencies  bet#een 
indexing  and  other  operations.  However,  under  the  index 
field  scheme,  if  we  encode  the  index  indication  differently 
for  different  operations  (i.e.,  we  invent  an  "operation  n 
indexed”  op-code) ,  then  scheme  1  wins  again.  (This  is  an 
indication  that  the  cost  of  a  conditional  encoder /decoder  of 
order  m  cannot  be  characterized  by  m  alone.  With  a  snail 
op-code  set,  the  additional  burden  on  the  enco der/decoder 
will  be  small.)  Furthermore,  by  combining  these  fields,  the 
possible  discrepancy  between  the  information  content  and  the 
minimum  redundancy  encodings  of  the  fields  is  lessened. 


-80- 


Should  there  be  an  unconditional  branch  instruction,  or 
should  each  instruction  indicate  its  successor?  The  answer 
here  is  identical  to  the  answer  for  indexing,  and  for 
identical  reasons.  With  variable-length  instructions,  it  is 
spatially  more  economical  to  include  a  branch  address,  or 
indication  that  no  branch  is  required,  with  each  instruction 
than  to  use  an  unconditional  branch  instruction.  One  never 
needs  more  than  one  consecutive  unconditional  branch,  so  the 
extra  power  in  the  latter  scheme  is  wasted.  (A.  branch  table 
should  not  be  a  table  of  branch  instructions,  but  of  branch 
addresses.  See  Section  4.2.) 

One  is  tempted  to  ask  at  this  stage  whether  other 
instructions,  such  as  add,  should  become  subfields  of  ever- 
grander  instructions.  But  the  ability  to  use  two  or  nore 
consecutive  adds  is  useful,  so  the  subfield  schema  is 
inadequate.  If  we  allow  the  number  of  subfields  in  a  given 
instruction  to  be  arbitrary,  then  we  are  simply  switciing 
terminology:  these  subfields  would  be  an  instruction 


sequence. 
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5  variable  DATA 

The  previous  chapter  considered  quantizations  and 
encodings  suitable  for  the  instructions  and  constants 
appearing  in  a  program.  Knowing  the  value  of  each  constant, 
we  can  allot  it  the  appropriate  space.  Regardless  of  rhe 
space  chosen  for  a  variable  data  item,  the  possibility  of 
waste  or  overflow  may  arise  whenever  the  variable  chaages 
value.  The  usual  ’’solution"  to  the  replacement  problem  for 
numbers  is  to  inform  the  programmer  when  overflow  occurs  and 
terminate  execution,  or  to  provide  the  opportunity,  but  not 
the  ability,  to  deal  with  the  situation  (PL/I  "ON" 
condition).  The  IBP!  1401  simply  overwrote  the  following 
data  item.  Clearly  these  "solutions’*  are  inadequate.  In 
this  section  we  present  two  data  organization  models  that 
save  space  and  greatly  reduce  the  probability  of  overfLow. 
Evaluation  of  these  models,  in  Sections  5.2  and  5.3,  alLows 
us  to  find  the  best  representation  of  variables  within  each 
model,  given  a  distribution  of  value  lengths,  and  cost 
function. 

For  character  strings  (and  arrays  with  variible 
dimensions)  the  replacement  problem  has  sometimes  been 
solved  by  the  use  of  fixed-length  descriptors,  describing 
the  length  and  location  of  a  string  (or  array)  in  a  fiiced- 
length  "free  area",  with  periodic  or  continual  space 


reclamation  (garbage  collection)  (Knuth  68  p.406).  Although 
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the  length  is  freed  to  some  extent  by  this  scheme,  it  is 
confined  to  a  fixed  range,  and  the  total  length  of  all 
strings  is  confined  to  the  length  of  the  free  area.  These 
two  constraints  are  the  result  of  fixing  the  length  of  a 
descriptor.  If  the  free  area  is  large,  much  of  it  will  be 
unused  at  any  one  time,  wasting  space;  if  it  is  small,  space 
reclamation  frequency  will  be  high,  wasting  time. 

The  descriptor  scheme  has  another  drawback.  The  space 
overhead  for  an  extra  address  and  for  unused  free  space,  and 
the  time  overhead  for  an  extra  memory  access  and  for  space 
reclamation  are  small  relative  to  large  data  items,  but 
enormous  relative  to  small  ones.  If  we  want  a  unified  data 
access  scheme,  we  must  allow  the  number  of  levels  to  vary, 
being  zero  for  values  which  fit  into  a  small  number  of  bits, 
and  one  or  more  for  longer  values. 

Variables  may  be  divided  into  two  classes  according  to 
the  form  of  declaration; 

1.  infinite  range 

e. g . ,  declare  i  integer 

declare  s  string  of  character 
Most  machine  designs  make  liars  of  infinite  range 
declarations, 
finite  range 

e.g.,  declare  i  [ j  to  k] 

d eclar e  s  string  [1  to  n ]  of  character 


2. 
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A  machine  design  with  no  overflow  means,  for  this 
class  of  declaration,  that  j,  k  and  n  are  not 
bound  to  any  fixed  range. 

(In  some  languages,  such  as  SNOBOL  and  APL,  all  variables 
are  implicitly  declared  to  have  type  union  of  certain  base 
types,  and  the  programmer  is  not  provided  with  the  abiLity 
to  declare  a  variable  to  have  a  more  restricted  type.  The 
implicit  declaration,  however,  could  be  made  explicit,  with 
either  infinite  or  finite  range  as  above.) 

There  is  a  solution  to  the  replacement  problem  for 
variables  of  the  second  class  that  is  not  available  to 
variables  of  the  first,  namely,  allocate  to  each  variable 
space  sufficient  for  representing  any  value  in  the  specified 
range.  Then,  assuming  that  the  arithmetic  unit  can  handle 
any  length  of  operand,  we  are  guaranteed  that  a  range  error 
will  occur  before  (or  as  soon  as)  an  overflow. 

The  obvious  solution  for  finite  ranges  presented  anove 
wastes  space  if  the  values  of  variables  are  not  evenly 
distributed  throughout  their  ranges,.  For  a  large  class  of 
variables,  however,  such  as  loop  controllers  and  array 
indices,  values  may  be  evenly  distributed  in  their  ranges. 
This  possibility  is  in  no  way  contradictory  to  rhe 
expectation  that,  in  general,  small  values  are  more  common 
than  large  values,  since  ranges  which  are  near,  or  include. 
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0  may  be  more  common  than  ranges  which  do  not  include  and 
are  not  near  0. 

Specification  of  a  finite  range  for  a  variable  can  be 
useful  to  the  programmer  for  range  checking,  and  to  the 
compiler  for  allocating  space.  But  when  it  is  inconvenient 
or  impossible  for  a  programmer  to  provide  this  information, 
an  initial  allocation  of  space  must  be  made,  with  provision 
for  an  indirection  when  the  space  is  inadequate.  This  poses 
two  questions:  How  much  space  should  a  variable  be  allotted? 
and  How  do  we  use  the  space? 

Suppose  b  bits  have  been  allotted  to  a  variable.  Each 
of  the  2**b  patterns  can  be  used  to  represent: 

(a)  a  complete  value 

(b)  a  partial  value,  e.g.,  the  initial  portion  of  some 
values  whose  representations  are  more  than  b  bits 

(c)  a  pointer,  i.e.  ,  a  location  or  indirection 

(d)  a  partial  pointer,  e.g.,  the  index  into  a  hash 
table,  with  an  associative  search  to  complete  the 
indirection 

(e)  a  partial  value  and  a  pointer  or  partial  pointer, 
e.g.,  the  first  a  (<b)  bits  may  contain  value 
information,  and  the  remaining  b-a  bits  contain 
pointer  information 

The  number  of  representation  possibilities  is  overwhelming. 
At  one  extreme,  we  may  designate  all  2**b  patterns  as  values 
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or  partial  values,  with  an  associative  search  foe  the 
remainder  of  partial  values.  A.t  the  other  extreme,  all 
patterns  may  be  pointers  or  partial  pointers.  The  nore 
space  is  given  over  to  pointers,  the  more  often  an 
indirection  will  be  needed,  and  the  cheaper  an  indirection 
will  be. 

A  scheme  which  falls  mid-way  between  these  two  extremes 
(there  are  others  which  fall  "mid-way")  is  to  use  half  the 
patterns  for  value  information  only,  and  half  for  poiiter 
information:  a  single  bit  declares  "I  am  a  complete  vaLue" 
or  "I  am  a  pointer  or  partial  pointer".  We  arbitrarily 
choose  this  scheme  for  further  examination. 

P.  nice  feature  of  the  mid-way  scheme  is  that  it  caa  be 
extended  back  into  the  instruction.  That  is,  we  may  use  one 
bit  of  an  operand  to  indicate  "value"  or  "address"  rather 
than  including  this  information  in  the  op-code.  The 
uniformity  is  appealing,  but  artificial,  since  ia  an 
instruction  we  are  distinguishing  constants  from  variables, 
whereas  in  the  data  area  we  are  distinguishing  short  values 
from  long  ones.  Unless  constants  and  variables  are  about 
equally  common,  the  price  of  this  uniformity  is  some  aided 
redundancy  in  instructions.  The  suggestion  is  similar  to  an 
"indirect  bit",  but  with  a  difference.  The  usual  indirect 
bit  in  an  instruction  (e.g.,  IBM  7090,  PDP-1)  does  not 
decide  the  status  of  the  operand  at  hand  --  that  is  an 
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address.  It  decides  the  status  of  the  addressed  item,  vaich 
may  be  a  value,  or  address  with  indirect  bit.  Throughout  a 
chain  of  indirection,  each  indirect  bit  is  separated  by  one 
level  from  the  item  whose  status  it  determines. 

When  the  value  of  a  variable  becomes  too  long  for  the 
space  allotted  to  it,  we  have  a  choice  of  actions.  We  nay: 

(a)  allot  a  new,  larger  space,  and  put  a  pointer  (or 
partial  pointer)  in  the  old  space  to  point  to  the 
new;  i.e.,  add  a  level  of  indirection. 

(b)  allot  a  new,  larger  space,  and  change  the  politer 
(or  partial  pointer)  that  pointed  to  the  old  space 
to  point  to  the  new;  this  option  may  be  taken  only 
after  option  (a)  has  been  taken  one  or  more  tines, 
since  instructions  must  not  be  changed. 

Since  it  is  possible  (although  unlikely)  for  many  variables 
to  have  very  long  values  simultaneously,  we  cannot  be  sure 
that  the  size  of  the  old  space  in  choice  (a) ,  or  of  the  old 
pointer  in  choice  (b) ,  however  large,  is  sufficient  to 
contain  a  complete  pointer  to  the  new  space. 

Under  action  (a)  ,  the  new  larger  space  may  be: 

(i)  just  large  enough  to  accommodate  the  new  value;  we 
assume  that  the  value  describes  its  own  length, 
i.e.,  it  is  lef t-to-right  recognizable. 

(ii)  somewhat  larger  than  necessary,  to  allow  for 
further  growth  in  the  value  of  the  variable;  in 
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this  case  we  must  represent  both  the  length  o£  the 
value  and  the  length  of  the  space,  which  may  be 
expressed  relative  to  the  size  of  the  old  space. 

(iii)  some  fixed  relation  to  the  size  of  the  old  spice; 
the  length  of  the  space  (and  of  the  value)  may  be 
implied  by  the  level  of  indirection. 

Under  action  (b) ,  the  new,  larger  space  may  be  either  o£  (i) 
or  (ii)  above,  but  not  (iii)  .  When  the  value  of  a  variible 
becomes  shorter  again,  we  need  to  follow  the  indirection 
chain  only  until  we  reach  the  first  space  large  enough  to 
contain  it.  If  this  space  is  longer  than  necessary,  the 
surplus  may  either  be  returned  to  the  free  space  area 
(appropriate  with  choice  (i)  above)  ,  or  be  retained  for  < 

future  growth  (appropriate  with  choice  (ii) ) . 

The  variety  of  possible  solutions  to  the  replacenent 
problem  is  large.  Determining  the  best  solution  requires 
evaluation  of  each  solution  in  the  intended  environm 2 nt . 

For  example  evaluations,  we  shall  look  at  two  combinations: 
multi-level  indirection,  i.e. ,  choosing  action  (a)  at  each 
stage,  with  new  space  (iii)  exactly  double  the  old;  and 
zero/one-level  indirection,  i.e.,  choosing  action  (b)  as 
soon  as  possible,  with  new  space  (i)  just  large  enough.  In 
each  case  we  shall  determine  the  "best”  size  for  initially 
allotted  spaces. 
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5- 1  Fnvironment 

To  evaluate  a  treatment  of  variable  data  we  need  5 
statistical  distributions  representing  the  environme 
First,  we  need  to  know  how  many  variables  are  in  exists 
at  one  time-  For  a  given  program  this  changes  as  blocks 
procedures  are  entered  and  exited,  and  in  a  multiprograna 
environment,  as  programs  are  initiated  and  terminat 
Second,  we  need  the  distribution  of  lengths  of  the  values 
an  average  variable.  Third,  we  need  the  cost  per  unit  t 
of  occupying  a  data  location  as  a  function  of  the  locati 
We  may  prefer  instead  to  use  the  (space)  integral  of  t 
function-  fourth,  we  need  the  data  rate  between  memo: 
and  processors  as  a  function  of  the  location.  Finally, 
need  to  know  any  additional  charges  for  referencing  a  i 
location.  For  tapes  and  changeable  disks,  this  cost  w 
include  a  portion  of  any  mount  charges. 
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Figure  4:  Hypothetical  Distributions 
Representing  Variable  Data  Environment 
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For 

sections 

1. 

2. 


3. 


4. 


our  sample  environment  in  the  following 
we  shall  assume  the  following: 
that  the  number  of  variables,  n,  remains  const 
that  the  length  (number  of  bits,  characters 
other  unit  required  to  represent  the  value)  o 
variable  is  s  with  probability  f(s)  =  2**-s. 
that  the  occupancy  cost  per  time  is  proporti 
to  the  space  occupied. 

that  the  data  rate  between  memory  and  processo 
constant,  independent  of  the  location.  The 
rate  will  be  represented  by  the  data  path  wi 


d. 

5.  that  the  additional  cost  of  referencing  a 
location  is  zero,  independent  of  the  location. 


We  shall  assume  that  a  program,  whose  running  cost  we 

trying  to  minimize,  specifies  some  particular  number 

variable  references,  that  each  variable  reference  regu 

one  or  more  memory  references  as  described  in  the  model, 

that  1  in  4  variable  references  changes  the  value  of 

variable.  (This  ratio  of  value  changes  to  references 

typical  of  some  environments  (Alexander  72  p.41).) 

function  to  be  minimized  is: 

(cost  of  occupancy)  *  (mem  refs/vbl  ref) 

+  (mem  ref  cost)  *  (mem  refs/vbl  ref) 

which  is  proportional  to: 

(space  occupied)  *  (mem  refs/vbl  ref) 
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5.2  Multi-level  Indirection 

Let  the  initially  allocated  space  for  a  variable  of 
some  data  type  be  b  units  (bits,  characters,  etc.).  In  the 
multi-level  indirection  model,  if  the  value  of  a  variable 
fits  into  b- 1  units,  then  1  unit  is  used  to  say  so,  and  the 
value  occupies  the  remaining  b-1  units.  If  not,  then  1  anit 
says  not,  a  pointer  (or  partial  pointer)  occupies  the 
remaining  b-1  units,  and  2*b  units  are  allocated.  Repeated 
allocations,  each  double  the  previous,  are  made,  leaving  a 
trail  of  pointers,  until  the  value  fits  into  (2**k) *b  -  1 
units  for  some  k .  We  shall  determine  the  best  value  of  b 
for  this  model,  in  the  environment  described  in  the  previous 
section.  We  leave  management  of  the  area  used  for 
allocations  (subsequent  to  the  initial  allocation) 
unspecified,  but  suggest  that  the  "buddy  system”  (KnovLton 
65,  Knuth  68,  Purdom  S  Stigler  70)  may  be  particularly 
appropriate. 

The  space  required  for  a  variable  of  length  s  is: 

S  (s)  =  b  +  2*b  ♦  4*b  +  ...  ♦  (2**k)  *b 
for  the  smallest  integer  k  >  0  such  that  s  <  (2**k) *b  -  1 
The  average  space  for  all  variables  is: 


inf 

S  =  n  *  SUM  S  (s)  *f  (s) 
s=  1 
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(The  notation  "SUM"  is  used,  for  typographic  reasons,  in 
place  of  the  more  usual  capital  sigma,  and  "inf”  deaotes  an 
upper  limit  of  infinity.) 

The  number  of  memory  references  required  to  reference  a 
variable  of  length  s  is: 

R(s)  =  [b/d]  +  [  2*b/d  ]  ♦  [4*b/d]  +  ...  +  [  (2**k)  *b/I  ] 
where  [ ]  means  rounded-up  or  ceiling,  and  k  is  as  before. 

The  average  number  of  memory  references  per  variable 

reference  is: 

inf 

R  =  SUM  R  (s)  *f  (s) 
s—  1 

Appendix  F  is  a  graph  of  the  cost  per  variable  S*R/n  as  a 
function  of  the  initially  allocated  space  b,  for  various 
data  path  widths  d.  For  data  path  width  d  >  8,  the  cost 
function  is  minimized  by  choosing  initially  allocated  space 
b=4.  (For  d  <  8,  S*R  is  minimized  when  b=3,  4  or  5.) 

Furthermore,  for  b=4,  the  cost  function  is  virtually  Elat 
for  all  d  >  4.  This  means  that,  in  this  environment,  ifith 
this  model,  a  data  path  width  greater  than  4  is  a  waste  of 
money.  The  cost,  at  the  minimum,  is  less  than  the  cost 
incurred  by  a  model  that  allocates  6  units  per  variaale, 
with  data  path  width  6,  and  no  provision  or  cost  penalty  for 


overflow. 
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5.3  Zero/one- level  Indirection 

A  one-level  indirection  model  has  been  used  in  some 
well-known  systems  (XPL,  SNOBOL)  for  long  data  items  such  as 
character  strings.  The  data  address  in  an  instruction 
refers  to  an  initially  allocated  space,  which  contains  a 
pointer  to  the  value  in  a  "free  area".  When  the  length  of 
the  value  increases,  space  for  the  new  value  is  allocated 
sequentially  from  the  free  area  until  the  free  area  is  used 
up.  Then  compaction  (garbage  collection)  takes  place.  In 
this  section,  we  shall  adopt  this  model  with  two  important 
changes:  length  information  is  located  in  or  with  the  vaLue, 
rather  than  with  the  pointer  to  it;  and  a  short  value 
occupies  the  initially  allocated  space,  without  indirection 
into  the  free  area. 

Let  the  initially  allocated  space  be  b  units.  A  single 
unit  in  this  space  tells  whether  the  value  is  short  (£b-1 
units)  and  present,  or  long  (>b  units)  and  in  the  free  area. 
Let  the  size  of  the  free  area  be  u  addressable  units,  where 
an  addressable  unit  is  v  basic  units.  If  log(u)  >  b-1  then 
the  initial  space  is  not  sufficient  to  address  all  of  the 
free  area.  The  result,  as  described  previously,  is  that 
partial  pointers  must  be  completed  by  further  levels  of 
indirection  and/or  an  associative  search.  To  avoid  this,  we 
shall  adopt  the  constraint  log(u)  <  b-1. 


The  space  dedicated  to  variables  is: 
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S  =  n*b  +  u*v 

The  number  of  memory  references  per  variable  reference 
associated  with  variable  data  is: 

E  =  r  ♦  g*q 

where  r  is  the  average  number  of  memory  references  requited 
to  access  a  variable 

g  is  the  compaction  freguency  in  compactions  per 
variable  reference 

g  is  the  compaction  cost  in  memory  references  per 
compaction 

b-1  inf 

r  =  SUM  [  (s+1)  /d  ]*f  (s)  ♦  SOW  ([  b/d  ]♦[  s/d  ])  *f  (s) 
s=1  s=b 

=  (2-2**  (-[b/d  ]*d)  ) /(1-2**-d)  ♦  [  b/d  ]*2**  ( 1  - b)  -  1 
where  [  ]  means  rounded-up  or  ceiling  as  in  the  previous  section 
g  =  (proportion  of  unoccupied  free  space  used  each  rbl  ref) 

=  (average  new  length,  weighted  by 

prob(new  length  >  b  5  new  length  >  old  length)  j  = 

/  (average  unoccupied  free  space) 

If  we  may  assume  that  the  length  of  the  value  of  a  variable 

after  an  assignment  is  independent  of  the  previous  length, 

then 

(average  new  length,  weighted  by 

prob  (new  length  >  b  &  new  length  >  old  length)* 

inf  s-1 

=  SUM  SUM  f  (t)  *f  (s)  *s 
s=b  t=1 


(1  +  b)  *2**  (1-b)  -  (8/3)  *  (b+1/3)  *2**  (-2*b) 
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(average  unoccupied  free  space) 

inf 

=  u*v  -  n*SUM  [  s/v  ]*v*f  (s) 
s=b 

=  u*v  -  n*v*  ([  b/v  ]*2**  (1-b)  ♦ 

(2**  (-[  b/v  ]*v)  )  /(1-2**-v) 

To  calculate  q  we  assume  that  the  initial  unit  of  all  snort 
variables  must  be  accessed  once  to  find  out  they  are  short, 
and  that  all  long  variables  must  be  accessed  twice  to  read 
and  rewrite. 

b-1  inf 

g  =  n*  (SOM  1*f(s)  +  2*SUM  ([b/d  ]+[  s/d  ])  *f  (s) ) 
s=1  s=b 

=  n*(1  +  ( 8*[  b/d  ]  -  2)  *2**-b  + 

2**  (1-[  b/d ]*d)  /  (1-2**-d)  ) 

The  cost  function,  S*F  =  (n*b  ♦  u*v) * (r  +  g*q) ,  is 
minimized,  given  the  number  of  variables  n  and  the  data  path 
width  d,  by  choosing  the  free  space  u,  addressable  unit  v, 
and  initially  allocated  space  b  according  to  the  following 
table : 


n 

d 

u 

V 

b 

10 

1 

10 

1 

5 

>2 

11 

1 

5 

100 

1 

32 

2 

6 

2 

25 

3 

6 

3 

27 

3 

6 

>4 

28 

3 

6 

1000 

1 

60 

8 

7 

2 

62 

9 

7 

3-5 

64 

9 

7 

>6 

64 

10 

7 
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In  each  case,  the  initial  space  per  variable,  including 
"indirect  bit",  is  the  minimum  needed  to  address  the  free 
area.  As  might  be  expected,  the  cost  function  becomes 
virtually  flat  for  data  path  widths  greater  than  the  initial 
space  per  variable.  This  cost,  at  the  minima,  is 
approximately  egual  to  the  cost  incurred  by  a  model  that 
allocates  8  units  per  variable,  with  data  path  width  8,  and 
no  provision  or  cost  penalty  for  overflow. 

Although  the  conclusions  of  this  and  the  previous 
section  are  valid  only  for  one  artificial  length 
distribution  and  one  particular  cost  function,  the  purpose 
of  these  sections  has  been  to  illustrate  the  method  of 
evaluation,  and  demonstrate  that  certain  design  decisions 
can  profitably  be  based  on  knowledge  of  the  intended 


environment. 
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6  EXPERIMENTAL  results 

We  illustrate  some  of  the  ideas  presented  in  this 
thesis  by  choosing  an  environment  represented  by  a  class  of 
programs  written  in  a  high-level  source  language,  and 
comparing  a  conventional  machine-language  representation 
with  one  that  incorporates  some  of  our  suggestions.  We 
shall  transform  the  machine-language  in  stages,  so  that 
ideas  may  be  evaluated  individually.  In  Section  6.2,  we 
evaluate  a  particular  model  for  the  representatioa  of 
variables,  to  find  the  best  representation  within  the  model. 

Our  chosen  environment  is  compilation,  represented  by 
XCOM ,  a  compiler  for  the  XPL  language  to  IBM  360  macline- 
langiiage,  written  in  XPL.  This  choice  was  made  primarily 
for  convenience  --  some  statistics  on  XPL  programs  were 
already  available  (Alexander  72) ,  and  our  knowledge  of  the 
XPL  compiler  allowed  us  to  change  the  code  generation  to  our 
changed  machine-languages.  This  ability  was  important, 
since  the  intent  of  a  program  is  not  clear  from  an  IBM  360 
object  version:  it  is  difficult  or  impossible  to  separate 
the  computation  needed  to  overcome  the  inadequate 
architecture  from  the  computation  specified  by  the  source. 

Although  the  choice  was  expedient,  it  was  not 
unrealistic.  Since  XPL  resembles  PL/I,  the  results  may  be 
suggestive  for  compilers  written  in  that  langaige. 


Furthermore,  compilation  accounts  for  much  of  the  computer 
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time  at  many  installations.  Finally,  for  environments  jiite 
different  from  compilation,  and  languages  quite  different 
from  XPL ,  the  application  of  our  ideas  to  any  chosen 
environment  is  illustrated  as  well  by  this  example  as  by  any 
other,  and  that  is  our  primary  purpose. 

The  main  drawback  that  resulted  from  choosing  XCDd  as 
our  sample  was  that  the  data  definition  facilities  of  XPL 
cannot  adequately  describe  a  programmer's  intentions. 
Numeric  data  types  reflect  IBM  360  hardware,  a  trait 
inherited  from  PL/I,  and  although  bit(n)  is  available  in  rhe 
language,  it  was  well  known  by  the  users  that  only  bit {8), 
bit  (16)  and  bit  (32)  are  implemented  (variables  declare!  as 
bit(1)  were  taken  to  be  logical) .  There  are  no  constant 
definition  facilities  in  XPL.  A  macro  facility  can  be  used 
to  give  a  small  number  of  unstructured  constants  a  name,  but 
all  other  named  constants,  including  all  structured 
constants,  are  indistinguishable  from  variables.  A  second, 
less  annoying,  drawback  is  XCOM's  "solution”  to  the  limited 
number  of  registers  available  as  an  evaluation  stack  on  rhe 
IBM  360.  Father  than  emitting  machine  instructions  to 
overcome  the  inadequacy,  XCOM  gives  the  problem  back  to  the 
programmer  to  solve  by  writing  extra  source  statements. 
This  was  satisfactory  to  the  intended  users  of  XPL,  but  it 
means  that  our  versions  of  object  code  include  instructions 
whose  purpose  is  to  overcome  inadequacies  of  the  IBM  360. 
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In  the  following  sections,  we  have  used  stitic 
instruction  frequencies,  since  they  are  easier  to  obtain 
than  dynamic  frequencies.  We  are  therefore  minimizing  space 
rather  than  time.  But  the  work  of  others  (Wortman  72, 
Alexander  72)  suggests  that  dynamic  frequencies  are  similar. 

6.1  Instructions 

The  source  version  of  XCOM  (version  III.V,  uith 
COMPACTIFY)  is  a  2137-statement  XPL  program  occupying  4420 
lines.  A  few  minor  changes  to  XCOM  were  required  in  order 
to  compile  it  to  our  instruction  sets:  several  programming 
"tricks"  were  removed,  and  built-in  functions  and  variables 
(particularly  "file")  were  made  to  conform  to  the  rest  of 
the  language.  These  changes  are  listed  in  detail  in 
Appendix  G.  The  object  version  of  the  modified  source  is  an 
IBM  360  machine-language  program  occupying  486704  bits. 
This  figure  includes  nameless  constants,  and  constants  named 
using  the  macro  facility,  but  not  named  or  structured 
constants,  which  take  the  form  of  variables.  We  shall  call 
the  IBM  360  representation  of  modified  XCOM  "version  0".  In 
this  section  we  present  several  new  versions  which  employ 
some  of  the  techniques  for  removing  redundancy  developed  in 
Section  4. 

Our  first  version  of  XCOM,  version  1,  uses  a  machine- 
language  that  closely  expresses  the  capabilities  of  the  XPL 
language.  We  accept,  as  given,  the  design  decisions  in  SPL; 
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e.g.,  we  treat  input  and  output  as  addressable  variables, 
mod  as  an  operator,  and  length  as  a  call  on  a  system 
function.  We  assume  an  evaluation  stack,  and  one 
instruction  for  each  operator  in  XPL.  With  one  exception, 
all  instructions  are  zero-address  (polish  postEix) 
instructions,  and  are  obtainable  directly  from  a  canonical 
parse.  Data  addresses  are  in  lexic-level,  order-number 
format.  System  (predefined)  names  (variables,  procedures) 
have  lexic-level  0,  global  names  have  lexic-level  1,  etc. 
Branch  addresses,  as  suggested  in  Section  4.2,  are  expressed 
relative  to  the  updated  instruction  pointer. 


operation 


explanation 


push 

fetch 

store 

call 

return 

branch 

branch-on- false 


The  only  instruction  that  has  an 
explicit  operand.  Operand  may  be 
1 ogical-const ant,  number-constant, 
ch a racter-st ring- const ant,  branch- 
constant,  or  address.  Operand  is 
pushed  onto  the  stack. 

An  address  from  the  top  of  the  stack  is 
consumed.  The  addressed  value  from  the 
data  area  is  placed  on  the  stack. 

The  next-to-top  value  on  the  stack 
is  stored  at  the  address  on  top  of  the 
stack.  The  address  is  consumed,  the 
value  remains. 

Control  is  transferred  to  the  address 
on  top  of  the  stack.  The  address  is 
consumed.  The  return  address  is  saved 
(mechanism  unspecified) . 

Control  is  transferred  to  the  retura 
address. 

Control  is  transferred  according  to  the 
relative  branch  value  on  top  of  the 
stack.  The  branch  value  is  consumed. 

If  the  logical  value  in  the  next-to- 
top  stack  position  is  false,  control  is 
transferred  according  to  the  relative 
branch  value  on  top  of  the  stack; 
otherwise  control  continues  unaltered. 
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pop 

or 

and 

not 

equal 

less 

greater 

not-egual 

less-or-egual 

great er-or-equal 

concatenate 

add 

subtract 

negate 

multiply 

divide 

modulo 

number- to- st ring 
index 


In  either  case,  both  the  branch  value 
and  logical  value  are  consumed. 

The  top  value  on  the  stack  is  consumed. 

~ T 


The  operand (s)  is  (are)  taken  from 
the  stack,  and  the  result  is  placed 
on  the  stack. 


_j 

The  address  on  top  of  the  stack  is 
modified  according  to  the  numeric  vaLue 
in  the  next-to-top  position.  Both  the 
address  and  numeric  value  are  consumed, 
and  the  modified  address  is  placed  01 
the  stack. 


Most  of  the  above  instructions  leave  no  ambiguity  about 
the  types  of  their  operands.  For  those  that  do,  we  incLude 
information  in  the  op-code  to  distinguish  the  type  of 
operands.  For  "push",  this  information  must  be  present  in 
the  instruction.  For  the  others,  we  shall  consider  the 
effect  of  moving  type  information  to  operands  in  version  5. 


The  three  main  advantages  which  version  1  machine- 
language  enjoys  over  version  0  (IBM  360)  machine-language  in 
the  representation  of  XCOM  are: 

(a)  Information  concerning  the  lengths  of  operands  has  been 
removed  from  op-codes.  This  means  that,  in  Section 
6.2,  we  are  free  to  design  data  representations  which 
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suit  the  environment;  overflow/waste  in  data  is  not 
implied  by  the  instruction  set. 

(b)  Register  capabilities  have  been  made  uniform  (cf. 
divide,  mod,  multiply) ,  and  the  stack  regularity  allows 
register  specification  to  be  implicit. 

(c)  The  operations  available  are  those  of  the  XPL  languge. 

To  observe  the  effect  of  just  these  three  improvements,  we 
shall  allow  space  for  op-codes,  constants  and  addresses 
similar  to  version  0  (IBM  360)  allotments: 
op-code:  8  bits 

constant:  logical:  8  bits  (1  addressable  unit) 

number:  24  bits  (LA, LH , L  instructions) 

character- string:  8  bits  specifying  lengta 

in  characters,  plus 
8  bits  per  character 

branch:  24  bits 
address:  lexic-level:  4  bits 

order-number:  12  bits 

The  space  required  by  version  1  with  this  encoding  is: 
op-codes:  18462*8  =  147696  bits 
constants:  logical:  40*8  =  320  bits 

number:  1755*24  =  42120  bits 

character  strings:  214*8  ♦  3134*8  =  26784  bits 
branch:  889*24  =  21336  bits 
addresses:  lexic-level:  5514*4  =  22056  bits 

order-number:  5514*12  =  66168  bits 
total:  299696  bits  =  61.6%  of  version  0  space 
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6.1.1  Op-codes 

To  get  maximum  benefit  from  iterative  pairing,  m- 
tupling,  or  conditional  coding,  commutative  instructions 
should  be  placed  in  a  standard  order.  For  example,  the 
instructions  "push-number,  push-address,  fetch-number,  lid" 
could  equally  well  be  written  in  the  order  "push-address, 
fetch-number,  push-number,  add”  with  no  change  ia  effect. 
Every  commutative  operator  presents  an  opportunity  to 
exchange  the  sequences  of  instructions  that  develop  the  two 
operands.  We  have  arbitrarily  chosen  to  place  development 
of  a  complex  expression  operand  before  a  single  variable  or 
constant  operand,  and  we  have  chosen  to  place  a  variable 
operand  before  a  constant  operand.  Within  the  category 
"variable  operand"  we  placed  procedure  (function)  calls 
ahead  of  arrays,  and  arrays  ahead  of  simple  variables.  We 
realize  that  the  meaning  of  a  program  may  be  changed  if 
development  of  an  operand  involves  a  call  to  a  procedure 
with  side  effects,  but  we  choose  not  to  cater  to  such 
programming  practices. 

The  non-commutat i ve  relations  "less",  "greater",  "less¬ 
or-equal",  "greater-or-equal"  are  not  all  necessary.  We 
have  arbitrarily  chosen  to  replace  all  occurrences  of 
"greater"  by  "less"  with  exchanged  operands,  and  to  replace 
"greater-or-equal"  by  "less-or-egual"  with  exchaaged 
operands.  In  fact,  the  three  operators  "not",  "less",  and 
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"le ss-or-equal"  are  not  all  necessary;  similarly,  the  three 
operators  ’’not",  "equal”,  and  "not-equal”  are  not  all 
necessary.  However,  we  have  not  gone  to  the  extremity  of 
reducing  the  op-code  set  further,  merely  to  see  the 
reconstruction  of  the  eliminated  operators  by  combination. 

We  did  not  wish  to  bias  our  machine  comparisoa  by 
performing  any  code  optimizations  not  performed  by  X20M. 
Accordingly,  we  emitted  a  gratuitous  "return”  at  the  enl  of 
each  procedure,  whether  needed  or  not,  and  we  emitted  a 
"branch"  following  every  alternative  in  a  case  statement, 
whether  needed  or  not.  We  therefore  see  instruction  pairs 
such  as  "return,  return",  which  need  not  exist.  Since  XCOM 
performs  some  amount  of  constant  folding,  so  did  we, 
although  the  effect  was  negligible. 

Version  2  op-codes  are  an  open-ended  minimum  redundancy 
encoding  of  version  1  op-codes  under  the  assumption  of  op¬ 
code  independence.  Appendix  H  contains  the  frequencies  of 
version  1  op-codes.  Version  2  op-codes  require  66946  bits, 
an  average  of  3.6  bits  per  op-code,  or  45.3%  of  the  spare 
required  by  version  1  op-codes. 

Version  3  op-codes  are  a  conditional  encoding  of 
version  1  op-codes  (see  Section  4.4).  Rather  than  maintain 
context  whenever  possible  (see  Section  4.4.1)  we  established 
a  standard  context  at  every  instruction  that  is  the  object 
of  any  branch  or  call.  Our  result  is  thus  weakened,  but 
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easier  to  achieve. 

The  chosen 

standard  context  was 

an 

arbitrary 

tuple  of 

instructions 

.  Our  result 

might 

be 

improved  by 

choosing 

a  common,  naturally  occuring 

tuple 

of 

instructions  as  standard  context. 

conditional 

size 

bits 

percent 

perce l t 

cod ing 

of 

per 

of 

of 

order 

sample 

op-code 

version  2 

version 

1 

1 

66946 

3.6 

100 

45.3 

2 

38286 

2.  1 

57.2 

25.9 

3 

31486 

1.7 

47.0 

21.3 

4 

29313 

1.6 

43.8 

19.8 

Version  4  op-codes  are  an  iterative  pairing  of  version 
1  op-codes  (see  Section  4.4).  The  pairs  being  combined  oust 
not  include  any  pairs  whose  second  member  is  the  object  of  a 
branch.  A  stricter  condition,  which  we  found  none 
convenient,  is  to  disallow  pairs  of  instructions  which  cross 
a  statement  boundary.  When  our  initial  op-code  set  had 
increased  to  178  different  instructions,  we  terminated  the 
pairing  procedure,  since  at  this  point  our  ability  to  give 
sensible  names  to  the  new  instructions  was  exhausted. 
Appendix  I  lists  the  version  4  op-code  set  with  their  final 
frequencies.  An  open-ended  minimum  redundancy  encoding  of 
version  4  requires  33380  bits,  an  average  of  4.85  bits  per 
op-code  (1.8  bits  per  original  op-code),  49.9%  of  the  space 
required  by  version  2  op-codes,  22.6%  of  the  space  required 
by  version  1  op-codes. 
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Version  5  op-codes  are  version  1  op-codes  with  type 
information  removed  (see  Section  4.4.2  and  Appendix  J)  . 
open-ended  minimum  redundancy  encoding  of  version  5  requires 
44338  bits,  an  average  of  2.55  bits  per  op-code,  66.2 i  of 
the  space  required  by  version  2  op-codes,  30.0%  of  the  space 
required  by  version  1  op-codes. 

We  have  presented  separate  versions  of  op-ccdes, 
demonstrating  open-ended  minimum  redundancy  encoding, 
conditional  coding,  iterative  pairing,  and  type  removal  in 
order  that  the  coding  gains  can  be  evaluated  individually. 
But  they  are  not  mutually  exclusive  techniques.  In  general, 
however,  the  benefit  derived  from  the  combination  of  these 
techniques  is  less  than  the  sum  of  their  separate  benefits. 

6.1.2  Constants 

The  XPL  language  does  not  have  a  logical  data  type. 
Instead,  the  even  and  odd  integer  values  double  as  false  and 
true  when  context  requires  a  logical  value.  We  considered 
the  variables  declared  as  ”bit(1)”  to  be  logical  variables, 
and  we  took  integer  constants  to  be  logical  when  required  by 
context.  Our  version  1  encoding  of  the  40  logical  constants 
in  our  sample  gave  each  constant  a  single  addressable  unit 
on  the  IBM  360,  i.e.,  8  bits.  When  the  addressable  unit  in 
the  instruction  stream  is  decreased  to  1  bit,  it  is  obvious 
that  logical  constants  may  be  encoded  in  12.5%  of  the  space 
used  in  version  1. 
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Of  all  uniform  number  schemes  (described  in  Section  4.1 
and  Appendix  B. 1)  the  scheme  that  best  matches  the 
distribution  of  integer  constants  in  our  sample  is  the  base 
two  scheme  that  uses  digits  -1  and  0,  with  interspersed  go- 
stop  bits.  This  encoding  requires  9985  bits,  an  average  of 
5.67  bits  per  integer,  23.7%  of  the  space  required  by 
version  1  integers.  Better  yet,  the  binary  sign-and- 
magnitude  scheme  of  Appendix  B. 2  requires  9648  bits,  5.50 
bits  per  integer,  22.9%  of  the  space  required  by  version  1 
integers.  A  minimum  redundancy  encoding  of  the  integers  in 
our  sample,  which  is  computationally  useless  but  indicates  a 
lower  bound,  requires  7022  bits,  4.00  bits  per  integer, 
16.67%  of  the  space  required  by  version  1  integers.  The  XPL 
language  does  not  include  any  numeric  data  types  other  than 
integers. 

The  remaining  data  type  is  character  strings,  but  for 
reasons  stated  in  Section  4.1  (standardization,  other 
studies) ,  we  make  no  attempt  at  improving  the  representation 
of  characters.  We  shall,  however,  choose  among  the  three 
methods  listed  in  that  section  for  delimiting  the  length  of 
the  string,  on  the  basis  of  our  sample  data.  Since  the 
average  number  of  characters  per  string,  14.6,  is  greater 
than  the  number  of  bits  per  character,  8,  we  prefer  to 
reserve  one  8-bit  pattern  not  used  to  represent  any 
character  as  a  stopper,  rather  than  append  a  go-stop  bit  to 
each  character.  If  we  encode  the  length  of  the  string 
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explicitly  using  the  scheme  of  Appendix  B.2  for 
compatibility  with  integers,  we  require  1494  bits,  or  7.0 
bits  per  string.  Thus  the  explicit  length  method  proves  to 
be  best. 

6.1.3  Addresses 

Since  version  1  to  4  op-codes  include  type  information, 
we  may  keep  the  addressing  spaces  for  each  type  separate, 
and  choose  the  appropriate  addressable  unit  of  data 
separately  for  each  type.  In  the  absence  of  nore 
restrictive  range  information  than  is  provided  by  the 
declaration  of  XPL  integer  variables,  we  decided  to  give  all 
XPL  integer  variables  the  same  amount  of  initially  allocated 
space.  Similarly  character  string  variables,  waose 
declarations  provide  no  range  information,  are  all  given 
equal  initial  spaces.  Logical  variables,  of  course,  all 
have  equal  initial  spaces.  The  distribution  of  lengths 
allotted  to  simple  variables  of  each  type  is  therefore 
constant,  and  is  the  appropriate  addressable  unit  of  data. 

Given  the  length  of  a  single  array  element,  the 
declaration  of  arrays  in  XPL  tells  us  their  lengths  exactly. 
From  the  distribution  of  array  lengths  and  reference 
frequencies,  we  may  calculate  the  best  addressable  unit, 
trading  alignment  space  for  address  space.  We  found, 
however,  that  in  our  sample  the  total  address  and  array 
space  for  each  data  type  was  not  very  sensitive  to  the 


-109- 


addressable  unit.  Furthermore,  the  amount  of  data  available 
was  not  statistically  significant. 

The  deepest  nesting  of  scopes  in  our  sample  gaze  us 
only  four  lexic-levels  to  encode.  The  representation  of 
Appendix  B.2  (non-negatives  only)  required  12652  bits,  an 
average  of  2.4  bits  per  lexic-level,  59.7%  of  the  space 
taken  by  the  version  1  encoding. 

We  found  that  order-numbers  were  encoded  by  the 
representation  of  Appendix  B.2  into  25925  bits,  an  average 
of  4.9  bits  per  order-number,  40.8%  of  the  space  taken  by 
the  version  1  encoding. 

6.1.4  Branching 

From  a  list  of  branch  distances  measured  in 
instructions,  and  the  results  of  the  preceding  sections,  we 
encoded  branch  addresses  as  follows:  Estimating  the  number 
of  bits  required  to  represent  an  average  branch  address,  we 
made  an  approximate  translation  of  branch  distances  from 
instructions  to  bits.  We  then  encoded  these  distances, 
revised  our  estimate  of  branch  address  space,  and  iterated. 
For  each  version  of  op-codes  and  each  branch  address 
representation,  convergence  came  in  one  or  two  steps. 

The  results,  for  each  version  of  op-codes,  are  listed 
below  for  two  number  schemes:  the  base  two  scheme  of 
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Appendix  B.2,  and  the  base  three  uniform  number  scheme  using 
digits  0,1,2  and  stopper. 


branch 

size 

bits 

percent 

address 

op-code 

of 

per 

o£ 

base 

version 

sample 

address 

versicn  1 

2 

2 

15902 

17.9 

74.  5 

2 

3  (m=2) 

15312 

17.2 

71.  3 

2 

3  (m=3) 

15214 

17.  1 

71.  3 

2 

3  (m=4) 

15180 

17.1 

71.  1 

2 

4 

15254 

17.2 

71.5 

2 

5 

15448 

17.4 

72.  4 

3 

2 

12859 

14.  5 

60.  3 

3 

3  (m=2) 

12601 

14.2 

59.  1 

3 

3  (m- 3) 

12469 

14.0 

58.  4 

3 

3  (ra=4) 

12465 

14.0 

58.  4 

3 

4 

12545 

14.  1 

58.  3 

3 

5 

12647 

14.2 

59.  2 

The 

base  four  uniform 

number 

scheme 

using 

digits  0, 

1,2,3 

with 

interspersed  go-stop 

bits  gave  approximately  the 

same 

results  as  the  base 

three 

scheme 

(1% 

worse) . 

if  it  h 

increasing  base,  the  results  became  progressively  worse. 


This  completes  instruction  encoding.  The  following 


table  lists  the  accumulated  results,. 
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branch 

size 

percent 

percant 

address 

op-code 

of 

of 

of 

base 

version 

sample 

version  1 

version  0 

2 

2 

159133 

53 

33 

2 

3  (m=  2) 

129883 

43 

27 

2 

3  (m  =  3) 

122985 

41 

25 

2 

3 (m=  4) 

120778 

40 

25 

2 

4 

124919 

42 

26 

2 

5 

136071 

45 

23 

3 

2 

156090 

52 

32 

3 

3  (m=2) 

127172 

42 

26 

3 

3  (m=3) 

120240 

40 

25 

3 

3  (m=4) 

118063 

39 

24 

3 

4 

122210 

41 

25 

3 

5 

133370 

45 

27 

From 

the  table,  we 

see  that 

our  techniques 

have  reduced 

the  space 

required  to 

represent 

our  sample 

of  machine- 

language  instructions  to  approximately  25%  of  the  IBM  360 
representation  (version  0) ,  and  to  approximately  40%  of  a 
language-directed  machine  representation  that  did  not  employ 
our  techniques  (version  1) . 


6.2  Variables 

A  one-level  indirection  scheme,  with  a  free  area  and 
storage  compaction  procedure,  is  used  for  character  strings 
in  the  IBM  360  version  of  the  XPL  compiler,  although  the  IBM 
360  instruction  set  limits  each  string  to  256  characters.  A 
zero-level  indirection  scheme,  i.e.,  a  direct  schema,  is 
used  for  numeric  variables,  with  no  provision  for  overfLow. 
We  shall  evaluate  the  zero/one-level  indirection  modal  of 
Section  5.3  for  numbers  and  strings,  to  find  the  best  size 
for  the  initially  allocated  space,  free  area,  and 
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addressable  unit  of  data.  For  this,  we  require  an 
environment  as  specified  in  Section  5.1. 

1.  There  are  88339  integer  variables  and  array 
elements  (used  portions  of  files  included,  emitter 
arrays  excluded),  and  840  string  variables  and  array 
elements  in  our  sample.  The  IBM  360  version  of  XCOM 
allocates  space  for  variables  statically.  Since  it  is 
not  the  purpose  of  this  section  to  compare  dynamic  and 
static  space  allocation,  we  shall  do  likewise. 
Therefore  the  number  of  variables  is  constant. 

2.  A  representation  of  integers  that  minimizes  the 
space  occupied  by  integer  variables,  and  the  length 
distribution  of  variables  according  to  this 
representation,  were  computed  from  a  complete  trace  of 
all  integer  values  that  arose  during  the  executinn  of 
XCOM  compiling  itself.  The  length  distribution  of 
character  string  variables  according  to  a 
representation  which  gives  each  character  8  bits,  *ith 
one  extra  character  used  either  to  describe  the  length 
or  as  a  stopper,  was  computed  from  a  trace  of  all 
string  lengths  that  arose  during  the  same  execution. 

3.  The  occupancy  cost  per  time  was  assumed  tn  be 
proportional  to  the  space  occupied. 
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4 .  The  data  rate  between  memory  and  processor  was 
assumed  to  be  a  constant,  represented  by  the  data  path 
width. 

5.  No  additional  costs  are  incurred  by  referencing 
any  location. 

The  cost  function  is  therefore  (space) *  (time)  ,  measured  in 
(bits) * (memory  references). 

For  the  trace,  we  made  the  assumption  that  assignment 
operations  are  evenly  distributed  throughout  the  execution. 
This  allowed  us  to  compute  the  average  time  each  value 
spends  in  memory  by  monitoring  only  assignments  (including 
those  found  in  loop  headings  and  parameter  passing) . 
Further  refinement  of  our  measurements  would  reguire  an 
assumption  regarding  the  relative  times  needed  to  perform 
the  operations  of  our  proposed  machines. 

6.2.1  Integers 

If  the  values  of  integer  variables  are  encoded 
according  to  the  base  three  uniform  number  scheme  using 
digits  0,1,2  and  stopper,  then  integer  values  reguire 
5.659*105  bits  on  average,  or  6.4  bits  per  variable.  The 
base  four  uniform  scheme  using  digits  0,1, 2, 3  with 
interspersed  go-stop  bits  fared  slightly  worse,  at  5.  685M05 
bits  (also  6.4  bits  per  variable),  as  did  the  base  two 
scheme  of  Appendix  B.2,  at  5.744*105  bits  (6.5  bits  per 
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variable)  .  With  increasing  base,  the  results  became 
progressively  worse.  Due  to  computing  expense,  we  did  not 
test  representations  using  digits  other  than  0  to  B-1  for 
some  positive  base  B. 

With  these  encodings,  we  found  the  distributioa  of 
lengths,  and  tabulated  all  length  changes  during  execition 
of  our  sample.  From  this  we  computed  the  average  number  of 
memory  references  per  variable  access,  storage  compaction 
frequency,  and  compaction  cost,  as  functions  of  initially 
allocated  space,  free  space,  and  addressable  unit  of  free 
space.  We  did  not  make  any  assumptions  of  leagth 
independence,  as  in  Section  5. 

For  the  base  three  scheme,  the  cost  function  is 
minimized  (at  least  locally)  by  choosing  the  initially 
allocated  space  to  be  16  bits,  free  space  2.8  bits  per 
variable,  with  the  addressable  unit  of  free  space  beinj  8 
bits.  The  cost,  at  the  minimum,  for  data  path  width  16  bits 
or  greater,  is  the  same  as  the  cost  of  a  scheme  which  uses 
22  bits  per  variable,  with  data  path  width  22  bits,  anl  no 
provision  or  penalty  for  overflow. 

For  the  base  two  scheme,  the  results  are  simiLar.  The 
cost  is  minimized  (at  least  locally)  when  the  initially 
allocated  space  per  variable  is  17  bits,  free  space  per 
variable  is  3.7  bits,  and  the  addressable  unit  of  free  space 
is  6  bits.  The  cost,  at  the  minimum,  for  data  path  width  17 
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bits  or  greater,  is  the  same  as  the  cost  of  a  scheme  which 
uses  25  bits  per  variable,  with  data  path  width  25  bits,  and 
no  provision  or  penalty  for  overflow. 

Contemporary  machines  commonly  give  integer  varianles 
at  least  22  to  25  bits  (see  Figure  2) .  They  are  already 
paying  as  much  in  running  costs,  according  to  our  cost 
function,  for  the  fixed-length  scheme  as  they  would  for  rhe 
zero/one-level  indirection  scheme  (with  appropriate  decoding 
and  space  reclamation  hardware) .  The  fixed-length  scheme 
forces  programmers  or  compilers  to  choose  among  the 
available  number  sizes,  and  overflow  occurs  if  the  value  of 
any  one  variable  ever  reguires  more  space  than  it  was 
initially  allotted.  Put,  for  the  same  cost,  the  zero/one- 
level  indirection  scheme  frees  the  programmer  and  compiler 
from  choosing  a  number  size,  and  overflow  occurs  only  when 
all  variables  simultaneously  have  values  such  that  the  total 
length  of  all  values  in  the  free  area  exceeds  the  free  area. 
With  a  proper  encoding  of  values,  i.e.,  one  that  gives  Long 
codes  only  to  uncommon  values,  the  probability  of  overflow 
is  greatly  reduced. 

6.2.2  Strings 

A  similar  computation  from  the  distribution  of 
character  string  lengths  and  length  changes  finds  the 
minimum  cost  when  each  string  is  allocated  2  characters 
initially,  with  free  space  9.4  characters  per  variable. 
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addressable  to  the  character.  With  increasing  data  path 
width,  the  cost,  at  the  minimum,  approaches  that  of  a  scieme 
which  gives  each  string  25  characters,  with  data  path  width 
25  characters,  and  no  provision  or  penalty  for  overflow.  ht 
data  path  widths  of  2,  4,  8  and  16  characters,  the  oost 
matches  fixed-length  schemes  of  68,  44,  33  and  27  characters 
respectively. 

If  each  character  is  8  bits,  then  the  initial  space  per 
variable,  and  therefore  data  addressing  unit,  and  the 
addressable  unit  of  free  space  are,  by  coincidence,  the  same 
for  character  strings  and  integers.  In  each  case,  the 
initial  space  is  the  minimum  required  to  address  the  free 
space . 

6.2.3  Miscellaneous 

The  626  logical  variables  and  array  elements  obviously 
require  626  bits. 

Since  our  proposed  machines  use  a  "branch-indirect" 
instruction  with  data  address  for  case  statements  and  go  to 
statements,  we  include  the  branch  constants  associated  with 
these  instructions  in  this  section.  The  7  case  statemsnts 
and  4  go  to  statements  in  our  sample  require  a  total  of  157 
branch  constants  in  the  data  area.  The  longest  of  these 
jumps  over  4250  version  2,  3  or  5  instructions  (1603  version 
4  instructions) ,  or  approximately  28000  bits  of  code. 


If  we 


-117- 


allow  each  of  the  branch  constants  the  maximum  space  needed 
by  the  longest  one  according  to  the  encoding  of  Section 
6.1.4,  then  23*157  =  3611  bits  are  required. 

When  type  information  is  removed  from  instructions, 
data  and  procedures  must  be  self-describing.  In  our  sample, 
the  following  entities  occurred  with  the  indicated 
frequencies: 


logical  variables 

7 

integer  variables 

293 

string  variables 

48 

branch  constants (go  to) 

4 

logical  arrays 

4 

integer  arrays 

50 

string  arrays 

7 

branch  arrays  (case) 

7 

procedures 

86 

Self-identification  therefore  requires  993  bits,  or  1.96 
bits  per  item,  a  mere  4%  of  the  space  saved  by  removing  type 
information  from  instructions  (see  Section  6.1.1). 

6 . 3  Summary  of  Experimental  Results 

From  our  experiments  on  one  particular  sample  of 
machine-language  instructions  and  variable  data,  considering 
only  the  effect  on  memory  occupancy,  we  find  that: 

1  The  length  of  instructions  should  be  allowed  to 
vary  to  the  bit. 

Op-codes  may  be  encoded  in  an  average  of  3.6  t>its 
each,  or  may  be  conditionally  coded  or  combined  to 
further  decrease  space  requirements  to  less  thin  2 
bits  per  original  op-code. 


2 
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3  Integer  constants  may  be  encoded  in  an  average  of 
5.5  bits  each. 

4  Data  addresses  may  be  encoded  in  an  average  of  7.3 
bits  each. 

5  Branch  addresses  may  be  encoded  in  an  average  of 
14  bits  each,  even  with  instruction  lengths 
variable  to  the  bit. 

6  Using  the  zero/one-level  indirection  model  and 

cost  function  as  specified  in  Section  6.2,  intagar 
variables  should  be  given  16  bits  each,  with  an 
additional  2.8  bits  of  free  space  per  variable, 
incurring  a  running  cost  equivalent  to  that  of  a 
22  bit  fixed-length  scheme. 

7  Using  the  zero/one-level  indirection  model  and 

cost  function  as  specified  in  Section  5.2, 
character  string  variables  should  be  given  2 
characters  each,  with  an  additional  9.4  characters 
of  free  space  per  variable,  incurring  a  running 
cost  equivalent  to  that  of  a  25  character  fiiced- 
length  scheme. 
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7  CONCLUSION 

7 . 1  Summary 

A  machine  designer  who  is  charged  with  the  task  of 
designing  information  representations,  and  fast,  inexpensive 
circuitry  to  process  information  in  those  representations, 
will  naturally  choose  fixed  information  widths.  Any 
variability  in  information  widths  will  increase  either  the 
time  required  to  recognize  and  process  the  information,  or 
processor  complexity,  and  therefore  expense.  But  processor 
cost  and  decoding  speed  are  only  one  part  of  system  expense 
and  performance,  a  part  whose  importance  is  often  small,  and 
is  currently  decreasing.  Of  equal  or  greater  importance  are 
the  amount  of  memory  used  to  store  information,  the  speed 
with  which  information  moves  through  fixed  data  path  widths, 
and  the  cost  of  overflowing  some  hardware  limitation. 

By  representing  information  in  a  manner  which  suits  the 
information  being  represented,  we  gain  the  following 
advantages: 

1  Reduction  in  memory  space  required  to  represent 
program  and  data 

2  Reduction  in  execution  time,  due  to  more  efficient 
use  of  available  bandwidth 

3  Elimination  of  all  forms  of  overflow  from  macaine- 
language: 

(a)  ever-expan dable  operation  set 
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PO  unbounded  program  size 

(c)  infinite  range  for  constant  data 

(d)  infinite  addressing  space  for  variable  data 

4  Increased  range  of  variable  data-  The  range  of 
variable  data  need  not  be  limited  by  the  space 
initially  allocated  to  them.  This  freedom  may  be 
attained  without  increasing  running  costs. 

5  Freedom  to  make  hardware  changes  witiont 
consequences  at  the  machine-language  level.  ihen 
new  technology  improves  the  speed  of  a  device, 
hardware  widths  may  be  adjusted  freely  to  maintain 
the  balance. 

To  illustrate  these  points,  we  identified  dependencies 
and  contributors  to  redundancy  commonly  found  in 
instructions  generated  from  high-level  source  programs,  and 
presented  methods  for  encoding  instructions  to  satisfy 
points  1,  2  and  3  above.  We  listed  a  range  of  possibilities 
for  encoding  variable  data  to  satisfy  point  4,  and  evalaated 
two  schemes  in  an  example  environment. 

We  applied  some  of  the  ideas  presented  in  the  thesis  to 
the  instructions  generated  from  a  large  XPL  program,  anl  to 
the  values  of  variables  generated  during  execution  oE  the 
program.  For  this  sample,  we  were  able  to  trim  75%  from  the 
space  taken  by  a  contemporary  machine  representation,  and 
60%  from  the  space  taken  by  a  language-directed  machine 
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representation  that  did  not  employ  our  techniques.  The  cost 
of  a  data  organization  that  greatly  reduces  the  probability 
of  overflow  compared  favourably  with  a  conventional 
organization. 

7.2  Future  Research 


The  uniform  number  schemes  of  Section  4.1  and  Appeidix 
B. 1  are  a  generalization  of  the  radix  complement 
representation  suitable  for  variable-length  encodiigs. 
Different  digit  sets  may  result  in  different  carry 
properties,  and  therefore  affect  the  design  of  hardware  to 
perform  arithmetic.  Also,  one  may  develop  a  general 
algebraic  function,  with  the  digit  set  as  parameter,  foe  the 
number  distribution  matched  by  a  uniform  number  scheme. 


In  Section  5  we  did  not  pursue  data  organizations  that 
take  advantage  of  finite  range  declarations  of  variables. 
As  the  Pascal  language  and  its  offshoots  become  more 
popular,  such  data  organizations  will  become  more  useful. 
Also,  an  investigation  of  redundancy  in  the  structure  of 
complex  data  structures  (e.g.,  stacks,  trees,  networks)  may 
lead  to  a  better  representation  for  those  data  structures. 


The  examples  in  Section  6  used  static  instruction 
frequencies  to  minimize  space  requirements.  A  similar 
analysis  using  dynamic  frequencies  would  determine  to  what 
extent  space  minimization  and  time  minimization  could  be 
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achieved  simultaneously.  It  would  also  lead  to  some 
conclusions  concerning  the  performance  value  of  various 
device  widths  and  bandwidths. 

In  Section  6  we  made  space  quantization  decisions  for 
one  computing  environment.  Similar  analyses  for  other 
environments,  which  differ  from  each  other  in  controlled 
ways,  would  determine  to  what  extent,  and  in  what  ways,  the 
quantization  decisions  depend  on  each  environment  parameter. 
In  particular,  the  number  of  variables  in  our  enviroanent 
remained  constant  throughout  the  execution  of  a  program. 
Extending  the  analysis  to  cover  a  varying  number  of 
variables  would  considerably  widen  its  applicability. 

There  is  a  time  quantization  analogy  to  the  space 
quantization  considerations  in  this  thesis.  Such  a  study 
should  determine  the  appropriate  amount  of  asynchronism  in 
computer  operations. 
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A.PPENDIX  A:  Some  Information  Theorems 

In  this  appendix.  In  denotes  the  natural  (base  e) 
logarithm,  and  log  denotes  the  binary  (base  2)  logarithm. 
All  free  subscripts  are  understood  to  be  universally 
quantified  over  the  range  1,..,n,  and  all  sums  are  over  the 
range  1,..,n.  j  stands  for  an  (m-1) -tuple  of  subscripts. 
All  subscripted  variables  are  probabilities:  Pi  is  the 
probability  of  appearance  of  the  ith  symbol  in  any  position 
in  a  sequence  of  these  symbols;  Pij  is  the  probability  of 
appearance  of  the  ijth  m-tuple. 

Pj  >  0,  SUM  Pj  =  1,  SUM  Pij  =  SUM  Pjk  =  Pj 

j  i  k 

We  may  allow  Pj  =  0  if  we  accept  the  convention  0  log  0=0, 
0/0  =00=1 

Let  g  (m)  =  -  j  SUM  Pij  log  Pij 

m  i  j 

be  the  average  information  content  per  symbol  of  an  m-tuple 

of  symbols.  Then  the  estimate  of  entropy  based  on  m-tuple 

frequencies  is  G  (m)  =  N  g(m)  where  N  is  the  number  of 

symbols  in  an  average  message. 

Let  f  (m)  =  -  SUM  Pj  SUM  Pjk  log  Pjk 

j  k  Pj  Pj 

be  the  average  information  content  per  symbol,  conditional 
upon  the  preceding  m-1  symbols.  Then  the  mth  estimate  of 
class  entropy  based  on  conditional  symbol  entropy,  is 
F  (m)  =  N  f  (m)  . 

Define  d  (m+1)  =  -  1  SUM  Pijk  log  Pi j_P jk 

m+1  ijk  Pj 
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Lemma  1  (Ash  65) : 

-  SOM  Pi  log  Pi  <  -  SUM  Pi  log  Qi  with  equality  i£E  Pi  =  Qi 
Proof : 

In  x  is  convex,  lying  under  its  tangent.  Therefore 

In  x  <  x  -  1  with  equality  iff  x  =  1. 

ln(Qi/Pi)  <  Qi/Pi  -  1  with  equality  iff  Pi  =  Qi. 

Multiply  by  Pi  log  e  and  sum  over  i  to  obtain 

-  SUM  Pi  log  Pi  +  SUM  Pi  log  Qi 

<  log  e  (SUM  Qi  -  SUM  Pi)  =  log  e  (1  -  1)  =  0 
hence  the  result. 

Lemma  2: 

g(m+1)  <  d(m+1)  with  equality  iff  Pijk  =  Pij  Pjk/Pj 
Proof : 

SUM  Pij  Pjk  =  SUM  Pi j_Pj  =  1 
ijk  Pj  ~  ij  Pj 

hence,  by  lemma  1,  the  result. 

A. 1  Theorem: 

g  (m)  >  g  (m  +  1) 

g  (m)  =  g(m+1)  implies  g(m-1)  =  g(m) 

That  is,  g  may  begin  flat,  but  after  the  first  decrease, 

it  is  a  strictly  monotonically  decreasing  function  of  m. 

Proof  by  induction  on  m: 

Induction  basis: 

g  (1)  =  -  SUM  Pi  log  Pi 

=  -  1  SUM  Pik  log  Pi  -  1  SUM  Pik  log  Pk 
2  ik  2  ik 
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=  -(1/2)  SUM  Pik  log  (Pi  Pk) 

>  -(1/2)  SUM  Pik  log  Pik  by  lemma  1 

=  g  (2) 

note:  g(1)  =  g(2)  iff  Pik  =  Pi  Pk,  i.e.,  the  symbol 

probabilities  are  independent. 

Induction  step:  first  note  that 

(nn-1)  d  (m  +  1)  =  -SUM  Pijk  log  Pi j_P jk 

i  jk  Pj 

=  -  SUM  Pijk  log  Pi j  -  SUM  Pijk  log  Pjk 
ijk  ijk 

+  SUM  Pijk  log  Pj 
ijk 

=  2mg(m)  -  (m-l)g(m-l) 

(m+  1)  [  g  (m-1)  -  g  (m)  ] 

=  (m- 1)  [g(m-1)  -  g  (m)  ] 

♦  2mg(m)  -  (m-l)g(m-l)  -  (m  +  1 )  g  (m+ 1) 

=  (m-1)[g(m-1)  -  g(m)]  +  (m+ 1)  [  d  (m+ 1)  -  g(m+1)] 

The  first  term  is  non-negative  by  the  induction 
hypothesis.  The  second  term  is  non-negative  by  lemma 
2.  Hence  the  left  side  is  non-negative,  ani  the 
induction  is  complete.  It  can  be  seen  that  if  the  Left 
side  is  zero,  the  first  term  on  the  right  must  also  be 
zero,  hence  the  result. 
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A.2  Theorem: 

f  (m)  >  f  (m+1) 

f  (m)  =  f  (m+1)  iff  Pijk  =  Pij  Pjk/Pj 
Proof : 

f  (m)  =  -  SUM  Pj  SUM  Pjk  log  Pjk 
j  k  Pj  Pj 

=  -  SUM  Pjk  log  Pjk  +  SUM  Pj  log  Pj 

=  mg  (m)  -  (m-l)g(m-l) 

f  (m)  -  f  (m+1)  =  mg(m)  -  (m-l)g(m-l)  -  (m+1)  g  (m  +  1)  +  mg(m) 
=  (m  +  1)[g(m)  -  g(m+1)]  -  (m- 1)  [  g  (m- 1)  -  g(m)] 

By  Appendix  A.1  we  have 
(m+1)  [g  (m)  -  g  (m+1)  ] 

=  (m-1)[g(m-1)  -  g(m)  ]  +  (m+1)[d(m  +  1)  -  g(m  +  1|  ] 

therefore 

f(m)  -  f  (m  +  1)  =  (m+ 1 )  [  d  (m+1)  -  g(m+1)] 

>  0  by  lemma  2  with  equality  iff  Pijk  =  Pij  Pjk/Pj 

A.  3  Theorem:  f  (m)  <  g(m) 

Proof:  From  Appendix  A.  2  we  have  f(m)  =  mg(m)  -  (m-l)g(m-l) 

therefore  g(m)  -  f  (m)  =  (m- 1)  [  g  (m- 1)  -  g  (m)  ]  >  0 
by  Appendix  A.1. 


-127- 


APPENDIX  E:  Variable-length  number  encodings 


B. 1  The  base  four  family  of  uniform  number  schemes 


An  underscored  digit  or  seguence  of  digits  represeats  a 
repeating  digit  or  sequence  of  digits  which  extends 
infinitely  far  to  the  left.  A  parenthesized  number  is  the 
length  in  bits  of  the  representation  which  encodes  the 
initial  portion  as  shown  below,  and  gives  each  succeeling 
digit  two  bits,  with  interspersed  go-stop  bits  to  delimit 


the  length. 


(-3, -2, -1,0) 

(-2, -1,0,1) 

(-1,0,1, 

2) 

-12 

(7)  0-3  0 

(9)  0-110 

(9)0-1  1 

0 

-11 

(7)  0-2-3 

(9)0-1  1  1 

(9)  0-1  1 

1 

-10 

(7)  0-2-2 

(6)  0-2-2 

(9)0-1  1 

2 

-9 

(7)  0-2-1 

(6)  0-2-1 

(9)  0-1  2 

-1 

-8 

(7)  0-2  0 

(6)  0-2  0 

(9)0-1  2 

0 

-7 

(7)  0-1-3 

(6)  0-2  1 

(9)  0-1  2 

1 

-6 

(7)  0-1-2 

(6) 0-1-2 

(9)  0-1  2 

2 

-5 

(7)  0-1-1 

(6)  0-1-1 

(6)  0-1-1 

-4 

(7)  0-1  0 

(6)  0-1  0 

(6)  0-1  0 

-3 

(4)  0-3 

(6)0-1  1 

(6)0-1  1 

-2 

(4)  0-2 

(3)  0-2 

(6)0-1  2 

-1 

(4)  0-1 

(3) 0-1 

(3)  0-1 

0 

(3)  0 

(2)  0 

(2)0 

1 

(3) -3 

(3)0  1 

(3)0  1 

2 

(4)zl-2 

(6)  0  1-2 

(3)  0  2 

3 

(4)  1.3-  1 

(6)  0  1-1 

(6)  0  1-1 

4 

( 4) -3  0 

(6)  0  1  0 

(6)  0  1  0 

5 

(7) -3-2-3 

(6)  0  1  1 

(6)  0  1  1 

6 

(7) -3-2-2 

(9) 0  1-2-2 

(6)  0  1  2 

7 

(7)  -3-2-1 

(9)0  1-2-1 

(6)  0  2-1 

8 

(7)  -3-2  0 

(9) 0  1-2  0 

(6)  0  2  0 

9 

(7) -3-1-3 

(9) 0  1-2  1 

(6)  0  2  1 

10 

(7) -3-1-2 

(9)  0  1-1-2 

(6)  0  2  2 

11 

(7) -3-1-1 

(9)0  1-1-1 

(9)0  1-1- 

-1 

12 

(7) -3-1  0 

(9)0  1-1  0 

(9)0  1-1 

0 

representation: 

0-3:  101 

0-2:  110 

0-2:  10 

0-1:  111 

0-1:  11 

0-1:  11 

0  :  000 

0  :  00 

0  :  00 

-3  :  001 

0  1:  01 

0  1:  01 

-3-2:  010 

0  2:  10 

-3-1:  011 
-3  0:  100 


(0, 1,2,3) 

(7)  3  10 
(7)3  11 
(7)  312 
(7)313 
(7)  320 
(7)  321 
(7)  322 
(7)3  23 
(4)30 
(4)31 
(4)  32 
(3)3 

(3) 0 

(4) 01 
(4)  02 
(4)03 
(7)  010 
(7)  011 
(7)  012 
(7)  0  13 
(7)  020 
(7)021 
(7) 022 
(7) 023 
(7) 030 


30:  100 
31:  101 
32:  110 
3:111 
0  :  000 
01:  001 
02:  010 
03:  011 
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The  base  four  family  of  uniform  number  schemes,  continual 


0,2,3, 4) 

(2,3, 4, 5) 

(3,4,5, 6) 

(4,5, 6,7) 

(0,5,2,7| 

-12 

(9)  3244 

(9)  3244 

(7) 644 

(6)  644 

(8)  27250 

-11 

(6) 311 

(9)  3245 

(7) 645 

(6)  645 

(8)  2705 

-10 

(6) 312 

(9)  3252 

(7) 646 

(6)  646 

(8)  27252 

-9 

(6) 313 

(9)  3253 

(7)  653 

(6)  647 

(8)  2707 

-8 

(6) 314 

(9)  3254 

(7)  654 

(6)  654 

(5)  2720 

-7 

(6) 321 

(9)  3255 

(7) 655 

(6)  655 

(5)  27255 

-6 

(6) 322 

(6)  322 

(7) 656 

(6)  656 

(5)  2722 

-5 

(6) 323 

(6)  323 

(4)63 

(6)  657 

(5)  27257 

-4 

(6) 324 

(6)  324 

(4)  64 

(3)6  4 

(5)270 

-3 

(3)  31 

(6)  325 

(4)  65 

(3)  65 

(5)  2725 

-2 

(3)  32 

(3)  32 

(3)6 

(2)6 

(5)  272 

-  1 

(2)  3 

(2)3 

(3)  3 

(3)  67 

(‘0  27 

0 

(3)  34 

(3)34 

(4)34 

(6)  674 

<“)0 

1 

(6) 341 

(3)  35 

w  35 

(6)  675 

(5)  275 

2 

(6)  342 

(6)  342 

(4)36 

(6)  676 

(5)02 

3 

(6) 343 

(6)  343 

(7)  343 

(6)  677 

(5)  277 

4 

(6)  344 

(6)  344 

(7) 344 

(9)  6744 

(5)  27  50 

5 

(9)  3411 

(6)  345 

(7)  345 

(9)  6745 

(5)05 

6 

(9)  3412 

(6)  352 

(7) 346 

(9)  6746 

(5)  2752 

7 

(9)  3413 

(6)  353 

(7)  353 

(9)  6747 

<5)07 

8 

(9)  3414 

(6)  354 

(7) 354 

(9)  6754 

(8)  020 

9 

(9) 3421 

(6)  355 

(7)  355 

(9)  6755 

(8)  2755 

10 

(9) 3422 

(9)  3422 

(7)356 

(9)  6756 

(8)022 

1 1 

(9)  3423 

(9)  3423 

(7)  363 

(9)  6757 

(8)  2757 

12 

(9) 3424 

(9)  3424 

(7)  364 

(9)  6764 

(8)  27  70 

representation : 

2720 

1030 

27  25  5 

1001 

2722 

1010 

63:  011 

27257 

1011 

64:  100 

64:  00 

270 

1130 

31:  01 

65:  101 

65:  01 

2725 

1131 

32:  10 

32:  10 

6  :  110 

6:10 

272 

1110 

3  :  11 

3:11 

3  :  111 

67:  11 

27 

111  1 

34:  00 

34:  00 

34:  000 

0 

00  30 

35:  01 

35:  001 

275 

030  1 

36:  010 

02 

0310 

277 

001  1 

2750 

0130 

05 

0131 

2752 

0110 

07 

01 1  1 
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B.2  A  variable-length  sign-and-magnitude  number  scheme 


The  number  of  leading  Os  (after  the  minus  sign  if  present) 
tells  how  many  binary  digits  follow  constituting  the  vaLue. 


0:  11 

1:  01 
2:  0010 
3:  0011 
4:  000100 
5:  000101 
6:  000110 
7:  000111 
8:  00001000 


-1: 

101 

-2: 

10010 

-3: 

10011 

-4: 

1000100 

-5: 

1000101 

-6 : 

1000110 

-7: 

1000111 

-8: 

100001000 

This  scheme  is  positively  biased;  it  matches  a  distribution 
in  which  a  positive  integer  is  twice  as  common  as  the 
corresponding  negative  integer.  When  only  non-negative 
integers  are  required,  zero  may  be  represented  by  a  single 
bit.  When  only  positive  integers  are  required,  tie  dealing 


0  is  unnecessary. 
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AP PEND IX  C:  Data  address  and  variable  data  space  versus 
addressable  unit  of  variable  data  (see  Section  4.3) 


S  =  x*d  +  a*b  x  =  10.4  d  =  1*SUM  2*log(i*a) 

n  i=l 

inf 

a  =  SUM[s/b] *f (s)  f(s)  =  2**-s 

s=l 


S  -  2*x*log(n) 


data  to  be  5. 
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APPENDTX  D:  Iterative  pairing 

Suppose  that  in  a  representative  sequence  S  the  synbol 
pair  A1,A2  is  replaced  with  the  new  symbol  AO.  Let  Pi'  be 
the  probability  of  occurrence  of  Ai,  i=0 , . . , n  in  the  new 
sequence  S'.  Then 

P0«  =  P12/(1-P12),  PI*  =  (PI-PI 2) /( 1 -PI 2)  , 

P2 1  =  (P2-P12) /(1-P12)  ,  and  Pi'  =  Pi/(1-P12),  i=3,..,n. 

The  average  information  content  of  a  symbol  in  S  is 
I  =  -  SOM  Pi  log  Pi 

The  average  information  content  of  a  symbol  in  S'  per 
symbol  in  S  is 

n 

I'  =  -  ( 1-P 1 2)  SUM  Pi' log  Pi' 

i=0 

So  the  change  in  information  content  (per  symbol  in  S)  is 

I '-I  =  log  _ _ P1**P1  *  P2**P2  *  ( 1-P 12) **  (1-P12L 

(P1-P12) **  (P1-P12)  *  (P2-P12) ** (P2-P12)  *  PI 2 **P12 

note:  0  <  P12  <  P1rP2  <  1  so  the  fraction  is  always  positive. 

If  PI  or  P2  =  0  or  1  then  I'  =  I. 

If  PI 2  =  P1P2  then  I»  =  I. 

If  P12  <  P1,P2  «  1  then  by  applying  a  binomial  expansion 

we  find  that,  to  first  order,  I'-I  =  P12  log (P1P2/P12) . 

If  A1 ,A1  is  the  symbol  pair  replaced,  then 

I'-I  =  log  P1**P1 _ 1 _ 

(P1-2P11)  ** (P1-2P1 if  *  P11**P11 


or,  to  first  order,  I'-I  =  P11  log(Pl2/P11) 
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In  iterative  pairing,  we  combine  at  each  stage  that  pair  for 
which  I'-I  is  greatest.  Perhaps  a  good  heuristic  is 
choosing  that  pair  ij  for  which  Pij  is  greatest. 

If  the  symbols  A1  and  A2  appear  only  in  the  pair  ^1,A.2  then 
PI  =  P2  =  PI 2  and 

I'-I  =  PI  log  PI  +  (1-PI) log  (1-PI)  <  0  (unless  PI  =  0) 
therefore  such  pairs  should  always  be  combined. 
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APPENDIX  E:  Indexing  under  independence  assumptions 

Suppose  that  op-code  i  appears  with  probability  Pi 
independent  of  preceding  op-codes.  Suppose  also  that  a  lata 
address  in  an  instruction  is  indexed  with  probability  X 
independent  of  the  operation,  and  if  indexed,  that  index 
register  i  is  used  with  probability  Qi. 

Due  to  op-code  independence,  conditional  coding  does  not 
change  the  space  required  for  scheme  1  (including  an  index 
field  in  each  instruction) .  It  remains 

-  SOM  Pi  log  Pi  -  X  SUM  Qi  log  Qi  -  X  log  X  -  (1-X)  log  (1 -X) 
per  instruction. 

For  scheme  2  (include  an  indexing  instruction  to  modify  an 
address  in  the  following  instruction)  the  new  instruction 
sequence  contains  1+X  times  as  many  instructions.  Therefore 
prob  (OPi)  =  Pi/  (1  +  X) 
prob ( INDEX)  =  X/(1+X) 

For  pairs  of  instructions,  the  joint  probabilities  are 

prob (OPi,  OPj)  =  u  Pi  Pj 

prob (OPi,  INDEX)  =  V  Pi  X 

prob (INDEX,  OPj)  =  W  X  Pj 

prob (INDEX,  INDEX)  =  0 

where  U,  V,  and  V  are  determined  by  the  following: 

Since  op-code  i  must  be  followed  by  something, 

SUM  prob  (OPi,  OPj)  +  prob  (OPi,  INDEX)  =  prob  (OPi) 

therefore  U  Pi  +  V  Pi  X  =  Pi/(1  +  X) 
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Since  op-code  j  must  be  preceded  by  something, 

SUM  prob  (OPi,  OPj)  +  prob  (INDEX,  OPj)  =  prob(OPj) 
i 

therefore  U  Pj  +  W  X  Pj  =  Pj/(1+X) 

Similarly  an  INDEX  must  be  followed  and  preceded  by  something, 

therefore  W  X  =  X/(1  +  X)  and  V  X  =  X/(1  +  X) 

Finally,  all  probabilities  must  sum  to  1 

therefore  SUM  U  Pi  Pj  +  SUM  V  Pi  X  +  SUM  W  X  Pj  =  1 
•  •  •  • 

13  13 

hence  U  =  1- X  and  V  =  W  =  1 

HX  7+X 

The  op-code  space  per  original  instruction  becomes 

(1+X)  [-SUM  _Pi  SUM[1-X  Pi  Pj  1+X ]log[  VX  Pi  Pj  1  +  X] 

[  i  TTx  j  [  H-x  Pi]  [  1+X  Pi] 

-SUM  _Pi[  Pi  _X_  1±X]log[Pi  _X_  UX  ] 
i  1+X[  1+X  Pi]  [  1  +  X  Pi] 

-_X_  SUM[_X  Pj  1+X  ]log[ _X  Pj  1  +  X]] 
i7x  j  [1+X  X  ]  [1+X  “X  ]] 

The  space  for  the  operand  of  an  index  instruction  remains 

unchanged,  and  the  total  space  reduces  to  that  require!  by 


scheme  1. 


APPENDIX 
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initially  allocated  space 

Cost  per  variable  versus  initially  allocated  space, 

for  various  bandwidths ,  using  the  multi-level  indirection  model,  Section  5.2 
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APPENDIX  G:  Changes  in  XCOM  for  the  purpose  of 
generating  version  1  object  code 

1  In  12  statements  we  found  that  simple  variables  were 
subscripted  to  overlay  information  in,  or  obtain 
information  from,  the  preceding  or  following  storage 
locations.  8  of  these  were  removed,  4  were  changed. 

2  In  5  if  statements  a  numerical  value  was  used  where  a 

logical  value  is  normally  required.  These  *ers 
changed. 

3  In  1  statement  the  "pseudo-variable”  in.£!it(2)  was 

changed  to  a  new  "pseudo-variable"  input2. 

4  In  19  statements  the  2-dimensional  "pseudo-array"  file 
was  replaced  by  a  1- dimensional  array  (one  new  array 
for  each  of  the  three  "rows") ,  and  the  array  assignment 
converted  to  a  loop. 

5  The  single  instance  of  the  built-in  function  inline-  was 
removed. 

6  In  2  procedure  headers  no  type  was  declared,  aven 

though  a  value  is  returned.  In  both  cases,  type  was 
added. 

In  1  statement  the  built-in  function  byte  was  used  on 
the  left  of  an  assignment.  It  was  changed  to  a  simple 
string  assignment  by  using  a  concatenate  operator  on 
the  right  side. 


7 
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APPEFDIX  H:  Version  1  operation  frequencies 
operation  frequency  percent  of  total 

(label)  (986)  (not  included  in  total) 


push- logical- constant 

40 

0.22 

-number- const ant 

1744 

9.45 

-string-constant 

210 

1.14 

-  branch-constant 

889 

4.  82 

-  address 

5514 

29.87 

fetch-logical 

29 

0.16 

-number 

3077 

16.66 

-string 

295 

1.61 

-branch 

11 

0.06 

store-logical 

29 

0.16 

-number 

1011 

5.48 

-string 

159 

0.86 

call 

903 

4.89 

return 

145 

0.79 

branch 

449 

2.43 

branch- on- false 

451 

2.44 

pop-logical 

28 

0.15 

-number 

995 

5.39 

-string 

158 

0.  86 

or-logical 

21 

0.11 

-number 

5 

0.03 

and-logical 

15 

0.08 

-number 

43 

0.23 

not-logical 

5 

0.03 

-number 

0 

0 

equal -logical 

0 

0 

-number 

179 

0.97 

-string 

16 

0.09 

less-number 

83 

0.45 

-string 

2 

0.01 

not- equal- logical 

0 

0 

-number 

52 

0.28 

-string 

3 

0.02 

less-or-equal-number 

122 

0.66 

-string 

1 

0.01 

concatenate 

196 

1.06 

add 

302 

1.64 

subtract 

137 

0.74 

negate 

5 

0.03 

multiply 

5 

0.03 

divide 

8 

0.04 

modulo 

5 

0.03 

number- to- string 

57 

0.  31 

index-logical 

38 

0.21 

-number 

936 

5.07 

-string 

82 

0.44 

-branch 

7 

0.04 
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APPENDIX  I:  Version  4  operations  and  frequencies, 
obtained  from  version  1  operations  by  iterative  pairing. 

Explanation: 

The  instructions  are  a  subset  of  the  following: 

{zeroary,  unary-operand,  binary-operand-operand}  whecs 
"zeroary"  means  each  of  {call,  call-type,  return} 

"unary”  means  each  of  {branch,  return-type,  load-rype, 
not-type,  negate,  number-to- string} 

"binary"  means  each  of  {branch-on-f alse ,  store-non- destr l ctive-ty pe, 
store-destructive-type,  or-type,  and-type,  equal-type, 
not-equal-type,  add,  multiply,  less-type,  less-or-e gual- type, 
concatenate,  subtract,  divide,  modulo} 

"type"  means  each  of  {logical,  number,  string,  branch} 

"operand"  means  each  of  {const,  vbl,  array,  proc,  exp} 

"const"  means  operand  follows 

"vbl"  means  obtain  the  operand  from  the  address  which  foLlows 
"array"  means  obtain  the  operand  from  the  address  formed  by 
modifying  the  address  which  follows  according  to  the 
numeric  value  on  the  top  of  the  stack;  pop  the  stack 
"proc"  means  obtain  the  operand  by  calling  the  procedure 
whose  address  follows,  and  whose  parameters  are 
currently  on  the  stack 

"exp"  means  obtain  the  operand  from  the  top  of  the  stack; 
pop  the  stack 

Results  are  always  placed  on  the  stack. 
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instruction 

frequency 

pecce 

(label) 

(986) 

(not  in 

branch-const 

438 

3.37 

-  vbl 

4 

0.06 

-array 

7 

0.  10 

branch-on-f alse-vbl-const 

6 

0.09 

-array-const 

19 

0.28 

-proc-const 

5 

0.07 

-exp-const 

421 

6.  12 

call -pure- procedure 

599 

8.  71 

-number- procedure 

5 

0.  07 

return 

114 

1.66 

return-logical-const 

13 

0.  19 

-exp 

5 

0.07 

-number-const 

1 

0.01 

-vbl 

4 

0.06 

-exp 

1 

0.01 

-string-const 

1 

0.01 

-vbl 

3 

0.04 

-exp 

3 

0. 04 

st ore -non -destructive -logical -const -vbl 

1 

0.  01 

-number- const- vbl 

12 

0.  17 

-array 

1 

0.01 

-vbl-array 

2 

0.03 

-proc-vbl 

1 

0.01 

-exp- vbl 

5 

0.07 

-string-con st- vbl 

1 

0.01 

s t or e-destructive- logical -const- vbl 

10 

0.15 

-array 

16 

0.23 

-exp-vbl 

1 

0.01 

-array 

1 

0.  01 

- number- const- vbl 

122 

1.  77 

-array 

99 

1.44 

- vbl-vbl 

103 

1.50 

-array 

120 

1.74 

-array-vbl 

56 

0.  81 

-array 

61 

0. 89 

- proc-vbl 

39 

0.  57 

-array 

20 

0.  29 

-exp-vbl 

342 

4.97 

-array 

28 

0.  4  1 

-string- const- vbl 

22 

0.  32 

-array 

3 

0.  04 

-vbl-vbl 

22 

0.  32 

-array 

4 

0. 06 

-array-vbl 

6 

0.  09 

-array 

8 

0.  12 

- proc-vbl 

15 

0. 22 

-exp-vbl 

77 

1.  12 

-array 

1 

0.01 
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load -number- const 

835 

12.  14 

-vbl 

1394 

20.  26 

-array 

269 

3.  91 

-proc 

60 

0.  87 

-string-const 

90 

1.  31 

-vbl 

90 

1.31 

-array 

29 

0.  42 

-proc 

2 

0.  03 

or-logical-exp-exp 

21 

0.31 

-number- proc- vbl 

1 

0.01 

-array 

2 

0.  03 

-proc 

2 

0.03 

and- logical- exp-exp 

15 

0.22 

-number-vbl-const 

17 

0.25 

-vbl 

3 

0.  04 

-array-const 

5 

0.07 

-vbl 

4 

0.06 

-proc-const 

6 

0.  09 

-exp-const 

8 

0.  12 

equal -number- vbl- con st 

55 

0.  80 

-vbl 

5 

0.07 

-array-const 

91 

1.  32 

-vbl 

6 

0.  09 

-proc-const 

6 

0.09 

-vbl 

8 

0.  12 

-proc 

3 

0.  04 

-exp-const 

4 

0.06 

-array 

1 

0.01 

-st ring- vbl- const 

11 

0.  16 

-vbl 

1 

0.  01 

-array-vbl 

4 

0.06 

not- equal- number- vbl- const 

11 

0.  16 

-vbl 

1 

0.01 

-  array-const 

25 

0.  36 

-vbl 

6 

0.09 

-proc-vbl 

4 

0.  06 

-proc 

4 

0.  06 

-  exp-const 

1 

0.  01 

-string- vbl- const 

1 

0.  01 

-array-array 

1 

0.01 

-proc-array 

1 

0.  01 

add-vbl-const 

212 

3.  08 

-vbl 

24 

0.35 

-array-const 

16 

0.23 

-array 

4 

0.06 

-proc-const 

2 

0.03 

-vbl 

19 

0.28 

-array 

1 

0.01 

-proc 

7 

0.  10 

-exp-const 

7 

0.  10 

-vbl 

5 

0.07 

-  proc 

5 

0.07 
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multi  ply- vbl -const 

5 

0.07 

less- number- const -vbl 

23 

0.33 

-array 

3 

0.  04 

-proc 

8 

0.12 

-exp 

4 

0.  06 

-vbl-const 

14 

0.20 

-vbl 

14 

0.20 

-array 

1 

0.01 

-proc 

1 

0.01 

-exp 

2 

0.03 

-array-con  st 

3 

3.  .04 

-vbl 

3 

0.  04 

-array 

2 

0.  03 

-proc-exp 

1 

0.01 

-exp-const 

1 

0.01 

-vbl 

2 

0.03 

-exp 

1 

0.01 

-string- vbl- vbl 

1 

0.01 

-proc-vbl 

1 

0.  01 

less-or-egual- number- const- vbl 

11 

0.  16 

-array 

2 

0.  03 

-exp 

1 

0.01 

-vbl-const 

41 

0.  60 

-vbl 

38 

0.  55 

-array 

1 

0.  01 

-proc 

1 

0.  01 

-exp 

20 

0. 29 

-array-const 

3 

0.  04 

-vbl 

1 

0.01 

- proc-vbl 

1 

0.01 

-exp-vbl 

2 

0.  03 

-string- vbl- proc 

1 

0,01 

concatenate- const- vbl 

3 

0.  04 

-array 

10 

0.  15 

-proc 

4 

0.06 

-exp 

27 

0.  39 

-vbl-const 

10 

0.  15 

-vbl 

6 

0.  09 

-array 

1 

0.  01 

-proc 

13 

0.  19 

-exp 

2 

0.03 

-proc-const 

2 

0.  03 

-vbl 

6 

0.  09 

-exp-const 

25 

0.36 

-vbl 

43 

0.63 

-array 

5 

0.07 

-proc 

16 

0.23 

-exp 

23 

0.  33 
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subtract-const-vbl 

2 

0.  03 

-array 

2 

0.  03 

-pr  oc 

1 

0.01 

-vbl-const 

48 

0.  70 

-  vbl 

37 

0.54 

-array 

7 

0.  10 

-  proc 

1 

0.01 

-array-const 

13 

0.  19 

-vbl 

2 

0.03 

-array 

1 

0.  01 

-proc-const 

8 

0.  12 

-vbl 

1 

0.01 

-array 

1 

0.  01 

-exp-const 

2 

0.03 

-vbl 

8 

0.  12 

-array 

1 

0.01 

-  proc 

1 

0.01 

-exp 

1 

0.01 

divide-vbl-const 

6 

0.  09 

-exp-const 

2 

0.  03 

modulo-vbl- const 

5 

0.07 

not-logical-vbl 

2 

0. 03 

-array 

2 

0.03 

-proc 

1 

0.  01 

negate-vbl 

4 

0.06 

-array 

1 

0.01 

number- to- st ring -vbl 

42 

0.61 

-array 

4 

0.06 

-  proc 

1 

0.  01 

-exp 

10 

0.  15 

Notes:  The  first  operand  of  the  "store”  operatinns  is  the 

value  to  be  stored,  the  second  is  the  destination 
address.  The  first  operand  of  ”branch-on-f alse”  Is  a 
logical  value,  the  second  is  the  relative  branch 
distance.  When  both  operands  of  a  binary  operator 
require  values  from  the  stack,  the  right  operand* s 
values  are  on  top.  The  typed  ’’call"  instruction  means 
call  and  pop  the  result.  The  "load”  instruction  is 
used  only  to  place  an  index  or  parameter  on  the  stack. 
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APPENDIX  J:  Typeless  operations 

This  appendix  lists  the  typeless  operations  created 
from  version  1  op-codes.  Type  information  is  not  removed 
from  the  "push”  instruction.  Indexing  is  implied  by  an 
array  operand. 


push- logical- constant 

40 

0.23 

-number- const ant 

1744 

10.02 

-string-constant 

210 

1.21 

-  branch-constant 

889 

5.11 

-address 

5514 

31.69 

zeroary- 1 

145 

0.83 

unary- 1 

5945 

34.17 

-2 

62 

0.36 

-3 

5 

0.03 

binary-1 

2169 

12.47 

-2 

210 

1.21 

-3 

140 

0.80 

-4 

124 

0.71 

-5 

84 

0.48 

-6 

52 

0.30 

-7 

43 

0.25 

-8 

8 

0.05 

-9 

5 

0.03 

-10 

5 

0.03 

-11 

5 

0.03 

The  instructions  were 

created  as 

follows: 

-1 

-2 

-3 

-4  -5 

zeroary 

return 

unary-logical 

pop 

not 

-number 

pop 

nun 

i-to-str  neg 

-string 

pop 

-branch 

branch 

-address 

f etch/call 

binary-logical- logical 

or 

and 

-branch 

br-false 

-address 

stn 

-number-number 

add 

equal 

subtract 

less-eqaal  less 

-address 

stn 

-string-string 

cat 

equal 

not-equal 

less  less-equa 

-address 

stn 

binary-number-number  cont'd: 

-6  -7 

-8 

- 

9 

-10 

-11 

not-equal  and  divide 

multiply 

modulo 

or 
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